2024-03-11T19:26:56.040 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-11T19:26:56.040 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/orig.config.yaml 2024-03-11T19:26:56.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T19:26:56.190 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591288' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 94a539707733325322964c6a187c6a1b41be1daf subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMG0+8nYDBqhMwFynqAG6lTn/Ot9MVEBFM4ASwNRpNepoPP8kYPiRMBWTdnPJTnboWySGIEy9Zr+/NNFrfvt0eA= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnfAqwAXOsOpPpgl8B2XQIvcBw4f+6y+1KDKP5b8IjAVgNtR0fSO30iXeGKFDXN+eYm8IdZQwi8zGhapmj3+y8= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBctvLN32amZUN1qWjgvHBMzZ3x/MOQzJaMHMLb915X1aNWK5RqDk31/K2Eunn5zqd1Xtc1FKA3eWadkj9zDooA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-11T19:26:56.190 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-11T19:26:56.191 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-11T19:26:56.191 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-11T19:26:56.193 INFO:teuthology.task.internal:Checking packages... 2024-03-11T19:26:56.215 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-11T19:26:56.215 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-11T19:26:56.215 INFO:teuthology.packaging:ref: None 2024-03-11T19:26:56.215 INFO:teuthology.packaging:tag: None 2024-03-11T19:26:56.216 INFO:teuthology.packaging:branch: squid 2024-03-11T19:26:56.216 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:26:56.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-11T19:26:56.441 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607.g94a53970 2024-03-11T19:26:56.442 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-11T19:26:56.449 INFO:teuthology.task.internal:no buildpackages task found 2024-03-11T19:26:56.449 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-11T19:26:56.455 INFO:teuthology.task.internal:Saving configuration 2024-03-11T19:26:56.474 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-11T19:26:56.481 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:26:56.507 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.869869', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMG0+8nYDBqhMwFynqAG6lTn/Ot9MVEBFM4ASwNRpNepoPP8kYPiRMBWTdnPJTnboWySGIEy9Zr+/NNFrfvt0eA='} 2024-03-11T19:26:56.532 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.867203', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnfAqwAXOsOpPpgl8B2XQIvcBw4f+6y+1KDKP5b8IjAVgNtR0fSO30iXeGKFDXN+eYm8IdZQwi8zGhapmj3+y8='} 2024-03-11T19:26:56.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.868639', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBctvLN32amZUN1qWjgvHBMzZ3x/MOQzJaMHMLb915X1aNWK5RqDk31/K2Eunn5zqd1Xtc1FKA3eWadkj9zDooA='} 2024-03-11T19:26:56.554 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-11T19:26:56.560 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-11T19:26:56.560 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-11T19:26:56.561 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-11T19:26:56.561 INFO:teuthology.run_tasks:Running task console_log... 2024-03-11T19:26:56.660 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd9a6e14160>, signals=[15]) 2024-03-11T19:26:56.661 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-11T19:26:56.667 INFO:teuthology.task.internal:Opening connections... 2024-03-11T19:26:56.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:26:56.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:26:56.759 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-03-11T19:26:56.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:26:56.847 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-03-11T19:26:56.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:26:56.935 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-11T19:26:56.942 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-03-11T19:26:56.960 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-03-11T19:26:56.960 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-03-11T19:26:57.016 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:26:57.017 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:26:57.018 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:26:57.018 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-03-11T19:26:57.042 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-03-11T19:26:57.060 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-03-11T19:26:57.060 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-03-11T19:26:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-03-11T19:26:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-03-11T19:26:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-03-11T19:26:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:26:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:26:57.118 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:26:57.118 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-03-11T19:26:57.142 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-03-11T19:26:57.159 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-03-11T19:26:57.160 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:26:57.218 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:26:57.219 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:26:57.219 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:26:57.219 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-03-11T19:26:57.242 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-11T19:26:57.252 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-11T19:26:57.258 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-11T19:26:57.259 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:26:57.261 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:26:57.263 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:26:57.279 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-11T19:26:57.285 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-11T19:26:57.285 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:26:57.317 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:26:57.319 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:26:57.649 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-11T19:26:57.729 INFO:teuthology.run_tasks:Running task kernel... 2024-03-11T19:26:57.813 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-11T19:26:57.814 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-11T19:26:57.814 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-11T19:26:57.814 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:26:57.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:26:57.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-11T19:26:58.050 DEBUG:teuthology.task.kernel:sha1 for client.0 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T19:26:58.051 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-11T19:26:58.051 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-11T19:26:58.068 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-361.el9.x86_64 2024-03-11T19:26:58.068 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T19:26:58.068 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-11T19:26:58.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2024-03-11T19:27:00.353 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-11T19:27:01.853 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-11T19:27:07.742 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.0 MB 00:05 2024-03-11T19:27:19.491 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-03-11T19:27:26.816 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-03-11T19:27:29.425 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 15 kB 00:01 2024-03-11T19:27:31.068 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-11T19:27:36.051 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-11T19:27:38.309 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-11T19:27:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 1 Package 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 375 M 2024-03-11T19:27:38.325 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-11T19:28:01.948 INFO:teuthology.orchestra.run.smithi107.stdout:linux-firmware-20240219-143.el9.noarch.rpm 16 MB/s | 375 MB 00:23 2024-03-11T19:28:01.949 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:28:01.950 INFO:teuthology.orchestra.run.smithi107.stdout:Total 16 MB/s | 375 MB 00:23 2024-03-11T19:28:01.950 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-11T19:28:02.048 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-11T19:28:02.048 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-11T19:28:04.437 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-11T19:28:04.437 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-11T19:28:10.979 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-11T19:28:11.180 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-11T19:28:11.264 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T19:28:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T19:28:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:28:12.670 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-11T19:28:12.966 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-11T19:28:12.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-11T19:28:13.257 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-11T19:28:13.257 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-11T19:28:13.260 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2024-03-11T19:28:25.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11 19:28:25 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-11T19:28:25.038 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2024-03-11 19:28:25-- 2024-03-11T19:28:25.038 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 12s 2024-03-11T19:28:25.038 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 321M in 11s (27.9 MB/s) 2024-03-11T19:28:25.040 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-11T19:28:25.040 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-11T19:28:26.977 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2024-03-11T19:28:27.031 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2024-03-11T19:28:27.111 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2024-03-11T19:28:34.652 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-11T19:28:50.850 INFO:teuthology.orchestra.run.smithi107.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-11T19:29:13.388 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-11T19:29:13.389 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-11T19:29:13.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-11T19:29:13.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-11T19:29:13.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-11T19:29:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-11T19:29:13.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-11T19:29:13.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-11T19:29:13.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-11T19:29:13.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-11T19:29:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-11T19:29:13.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-11T19:29:13.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-11T19:29:13.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-11T19:29:13.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-11T19:29:13.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-11T19:29:13.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-11T19:29:13.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-11T19:29:13.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-11T19:29:13.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-11T19:29:13.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-11T19:29:13.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-11T19:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-11T19:29:13.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-11T19:29:13.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-11T19:29:13.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-11T19:29:13.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-11T19:29:13.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-11T19:29:13.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-11T19:29:13.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-11T19:29:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-11T19:29:13.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-11T19:29:13.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-11T19:29:13.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-11T19:29:13.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-11T19:29:13.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-11T19:29:13.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-11T19:29:13.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-11T19:29:13.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-11T19:29:13.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-11T19:29:13.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-11T19:29:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-11T19:29:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-11T19:29:13.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-11T19:29:13.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-11T19:29:13.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-11T19:29:13.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-11T19:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-11T19:29:13.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-11T19:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-11T19:29:13.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-11T19:29:13.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-11T19:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-11T19:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-11T19:29:13.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-11T19:29:13.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-11T19:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-11T19:29:13.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-11T19:29:13.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-11T19:29:13.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-11T19:29:13.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-11T19:29:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-11T19:29:13.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-11T19:29:13.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-11T19:29:13.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-11T19:29:13.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-11T19:29:13.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-11T19:29:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-11T19:29:13.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-11T19:29:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-11T19:29:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-11T19:29:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-11T19:29:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-11T19:29:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-11T19:29:13.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-11T19:29:13.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-11T19:29:13.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-11T19:29:13.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-11T19:29:13.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-11T19:29:13.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-11T19:29:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-11T19:29:13.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-11T19:29:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-11T19:29:13.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-11T19:29:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-11T19:29:13.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-11T19:29:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-11T19:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-11T19:29:13.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-11T19:29:13.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-11T19:29:13.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-11T19:29:13.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-11T19:29:13.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-11T19:29:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-11T19:29:13.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-11T19:29:13.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-11T19:29:13.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-11T19:29:13.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-11T19:29:13.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-11T19:29:13.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-11T19:29:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-11T19:29:13.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-11T19:29:13.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-11T19:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-11T19:29:13.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-11T19:29:13.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-11T19:29:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-11T19:29:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-11T19:29:13.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-11T19:29:13.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-11T19:29:13.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-11T19:29:13.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-11T19:29:13.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-11T19:29:13.578 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-11T19:29:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-11T19:29:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-11T19:29:13.581 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-11T19:29:13.582 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-11T19:29:13.583 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-11T19:29:13.584 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-11T19:29:13.585 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-11T19:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-11T19:29:13.587 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-11T19:29:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-11T19:29:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-11T19:29:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-11T19:29:13.591 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-11T19:29:13.592 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-11T19:29:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-11T19:29:13.594 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-11T19:29:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-11T19:29:13.596 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-11T19:29:13.597 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-11T19:29:13.598 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-11T19:29:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-11T19:29:13.600 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-11T19:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-11T19:29:13.602 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-11T19:29:13.603 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-11T19:29:13.604 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-11T19:29:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-11T19:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-11T19:29:13.607 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-11T19:29:13.608 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-11T19:29:13.609 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-11T19:29:13.610 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-11T19:29:13.611 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-11T19:29:13.612 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-11T19:29:13.613 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-11T19:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-11T19:29:13.615 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-11T19:29:13.616 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-11T19:29:13.617 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-11T19:29:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-11T19:29:13.619 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-11T19:29:13.620 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-11T19:29:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-11T19:29:13.622 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-11T19:29:13.623 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-11T19:29:13.624 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-11T19:29:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-11T19:29:13.629 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:13.629 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-11T19:29:13.651 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2024-03-11T19:29:13.651 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-11T19:29:13.651 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-11T19:29:13.651 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-11T19:29:13.651 INFO:teuthology.orchestra.run.smithi107.stdout:for file in vmlinuz System.map config; do 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout: fi 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-11T19:29:13.652 INFO:teuthology.orchestra.run.smithi107.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-11T19:29:13.653 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-11T19:29:13.653 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:13.653 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-11T19:29:13.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-03-11T19:29:13.728 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-03-11T19:29:13.728 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-03-11T19:29:13.729 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:29:13.730 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-03-11T19:29:13.731 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-11T19:29:13.731 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:13.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-11T19:29:14.396 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-03-11T19:29:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-11T19:29:15.853 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-03-11T19:29:15.854 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-03-11T19:29:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.s7lsqf1omN 2024-03-11T19:29:15.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.s7lsqf1omN 2024-03-11T19:29:15.994 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.s7lsqf1omN 2024-03-11T19:29:16.136 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.s7lsqf1omN is 6KB 2024-03-11T19:29:16.156 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.s7lsqf1omN 2024-03-11T19:29:16.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-03-11T19:29:16.244 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-11T19:29:16.245 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-11T19:29:16.245 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-11T19:29:16.245 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-11T19:29:16.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:29:16.408 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-03-11T19:29:46.412 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-11T19:29:46.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:29:46.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:30:21.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-11T19:30:30.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:30:30.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:30:48.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-11T19:31:00.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:31:00.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:31:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-11T19:31:18.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:31:18.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:31:18.987 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-11T19:31:19.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:31:19.510 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-11T19:31:19.510 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-11T19:31:19.547 INFO:teuthology.orchestra.run.smithi107.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:31:19.547 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-11T19:31:19.547 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-11T19:31:19.547 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-11T19:31:20.548 DEBUG:teuthology.parallel:result is None 2024-03-11T19:31:20.549 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-11T19:31:20.557 INFO:teuthology.task.internal:Creating test directory... 2024-03-11T19:31:20.557 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:31:20.564 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:31:20.567 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:31:20.606 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-11T19:31:20.613 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-11T19:31:20.653 INFO:teuthology.task.internal:Creating archive directory... 2024-03-11T19:31:20.653 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:31:20.660 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:31:20.662 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:31:20.792 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-11T19:31:20.798 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-11T19:31:20.798 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:31:20.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:31:20.814 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:31:20.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:31:20.831 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:31:20.839 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:31:20.858 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:31:20.916 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:20.937 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:20.938 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:20.960 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:21.273 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:31:21.354 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-11T19:31:21.361 INFO:teuthology.task.internal:Configuring sudo... 2024-03-11T19:31:21.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:31:21.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:31:21.377 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:31:21.496 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-11T19:31:21.505 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-11T19:31:21.505 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:31:21.513 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:31:21.515 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:31:21.571 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:21.683 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:21.825 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:21.926 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:22.050 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:31:22.051 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:31:22.179 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:22.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:22.275 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:22.359 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:22.402 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:31:22.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:31:22.472 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:22.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:31:22.613 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:22.655 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:31:22.702 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:31:22.702 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:31:22.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-03-11T19:31:22.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-03-11T19:31:22.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-03-11T19:31:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:31:22.851 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:31:22.953 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:31:23.448 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-11T19:31:23.456 INFO:teuthology.task.internal:Starting timer... 2024-03-11T19:31:23.456 INFO:teuthology.run_tasks:Running task pcp... 2024-03-11T19:31:23.467 INFO:teuthology.run_tasks:Running task selinux... 2024-03-11T19:31:23.476 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-11T19:31:23.476 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-03-11T19:31:23.618 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-03-11T19:31:23.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-03-11T19:31:23.689 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-03-11T19:31:23.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-03-11T19:31:23.744 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-03-11T19:31:23.746 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-11T19:31:23.746 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-03-11T19:31:23.789 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-03-11T19:31:23.789 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-03-11T19:31:23.808 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-03-11T19:31:23.808 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-03-11T19:31:23.829 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-03-11T19:31:23.830 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi145.front.sepia.ceph.com': 'permissive'} 2024-03-11T19:31:23.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:31:23.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:31:23.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:31:23.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:31:23.930 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:31:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:31:23.958 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-11T19:31:23.959 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:31:24.045 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:31:24.077 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:31:24.115 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-11T19:31:24.125 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-11T19:31:24.145 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-11T19:31:24.146 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi122.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-11T19:37:14.765 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-03-11T19:37:14.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:37:14.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:37:14.920 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-11T19:37:15.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-11T19:37:15.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-11T19:37:15.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:37:15.167 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-03-11T19:37:15.237 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-11T19:37:15.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-03-11T19:37:15.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:37:15.327 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-03-11T19:37:15.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-03-11T19:37:15.399 INFO:teuthology.run_tasks:Running task clock... 2024-03-11T19:37:15.412 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-11T19:37:15.412 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:37:15.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:15.419 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:37:15.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:15.422 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:37:15.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:37:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:37:15.467 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:37:15.472 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:37:15.489 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:37:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-03-11T19:37:15.509 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-03-11T19:37:15.515 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:37:15.519 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-03-11T19:37:15.526 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:37:15.533 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-03-11T19:37:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:37:15.549 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:37:15.566 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:37:15.566 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:37:15.585 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:15.589 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:15.589 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-03-11T19:37:15.589 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.589 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.589 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.590 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.609 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.613 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-03-11T19:37:15.704 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-03-11T19:37:15.756 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:37:15.808 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:37:15.931 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:37:15.939 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:37:15.939 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-11T19:37:15.939 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.940 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.940 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.940 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:37:15.941 INFO:teuthology.run_tasks:Running task install... 2024-03-11T19:37:15.949 DEBUG:teuthology.task.install:project ceph 2024-03-11T19:37:15.949 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-11T19:37:15.949 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-11T19:37:15.949 INFO:teuthology.task.install:Using flavor: default 2024-03-11T19:37:15.955 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-11T19:37:15.955 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-11T19:37:15.955 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:37:15.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:37:15.958 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:37:15.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:37:15.959 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:37:15.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:37:16.106 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:37:16.106 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:37:16.111 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:37:16.111 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:37:16.254 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:37:16.255 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:37:16.322 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:37:16.322 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:37:16.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:37:16.329 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:37:16.330 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:37:16.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:37:16.351 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:37:16.351 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:37:16.358 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:37:16.358 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:37:16.362 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:37:16.363 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:37:16.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:37:16.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:37:16.426 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:37:16.450 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:37:16.450 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:37:16.499 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:37:16.506 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:37:16.565 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-03-11T19:37:16.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-03-11T19:37:16.571 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-03-11T19:37:16.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-03-11T19:37:16.591 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:37:16.724 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:37:16.799 INFO:teuthology.orchestra.run.smithi122.stdout:74 files removed 2024-03-11T19:37:16.808 INFO:teuthology.orchestra.run.smithi145.stdout:74 files removed 2024-03-11T19:37:16.813 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-03-11T19:37:16.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-03-11T19:37:16.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:37:16.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:37:17.172 INFO:teuthology.orchestra.run.smithi107.stdout:74 files removed 2024-03-11T19:37:17.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:37:17.451 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 251 kB/s | 82 kB 00:00 2024-03-11T19:37:17.520 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 200 kB/s | 82 kB 00:00 2024-03-11T19:37:17.822 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-11T19:37:17.904 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-11T19:37:18.226 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 163 kB/s | 82 kB 00:00 2024-03-11T19:37:18.261 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-11T19:37:18.313 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-11T19:37:18.832 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-11T19:37:18.843 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 19 kB/s | 10 kB 00:00 2024-03-11T19:37:18.963 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.7 MB 00:00 2024-03-11T19:37:19.477 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-03-11T19:37:20.308 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-11T19:37:20.439 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-03-11T19:37:20.446 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-11T19:37:21.665 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.0 MB 00:01 2024-03-11T19:37:22.189 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-11T19:37:23.375 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-03-11T19:37:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-03-11T19:37:27.760 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:02 2024-03-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 821 kB/s | 8.0 MB 00:10 2024-03-11T19:37:34.277 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-11T19:37:35.716 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 707 kB/s | 6.1 MB 00:08 2024-03-11T19:37:36.094 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-03-11T19:37:38.408 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-03-11T19:37:39.950 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 5.0 kB/s | 15 kB 00:03 2024-03-11T19:37:41.127 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-11T19:37:44.287 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-03-11T19:37:46.035 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:37:47.043 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:37:47.065 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:37:47.066 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:37:47.067 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:37:47.069 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:37:47.071 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:37:47.075 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:37:47.076 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:37:47.167 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:37:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:37:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:37:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:37:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:37:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:37:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:37:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:37:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:37:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:37:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:37:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:37:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:37:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-11T19:37:47.208 INFO:teuthology.orchestra.run.smithi107.stdout:Install 150 Packages 2024-03-11T19:37:47.209 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 24 Packages 2024-03-11T19:37:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:37:47.209 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 223 M 2024-03-11T19:37:47.209 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-11T19:37:48.322 INFO:teuthology.orchestra.run.smithi107.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-11T19:37:48.475 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:37:48.476 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:37:48.490 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:37:48.491 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:37:48.492 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:37:48.493 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:37:48.495 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:37:48.498 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:37:48.499 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:37:48.581 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:37:48.603 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:37:48.604 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:37:48.605 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:37:48.606 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:37:48.607 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-03-11T19:37:48.608 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:37:48.609 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:37:48.610 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:37:48.611 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:37:48.612 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:37:48.613 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:37:48.614 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:37:48.615 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:37:48.616 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:37:48.617 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-03-11T19:37:48.618 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-03-11T19:37:48.619 INFO:teuthology.orchestra.run.smithi122.stdout:Install 150 Packages 2024-03-11T19:37:48.619 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 24 Packages 2024-03-11T19:37:48.619 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:37:48.619 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 223 M 2024-03-11T19:37:48.620 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-03-11T19:37:48.755 INFO:teuthology.orchestra.run.smithi107.stdout:(2/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 7.0 MB/s | 5.3 MB 00:00 2024-03-11T19:37:48.838 INFO:teuthology.orchestra.run.smithi107.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.7 MB/s | 145 kB 00:00 2024-03-11T19:37:49.328 INFO:teuthology.orchestra.run.smithi122.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-11T19:37:49.537 INFO:teuthology.orchestra.run.smithi122.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 4.2 MB/s | 886 kB 00:00 2024-03-11T19:37:49.541 INFO:teuthology.orchestra.run.smithi107.stdout:(4/174): ceph-common-19.0.0-1607.g94a53970.el9. 13 MB/s | 20 MB 00:01 2024-03-11T19:37:49.581 INFO:teuthology.orchestra.run.smithi107.stdout:(5/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 704 kB/s | 886 kB 00:01 2024-03-11T19:37:49.620 INFO:teuthology.orchestra.run.smithi122.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.7 MB/s | 145 kB 00:00 2024-03-11T19:37:49.639 INFO:teuthology.orchestra.run.smithi107.stdout:(6/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 2.7 MB/s | 2.1 MB 00:00 2024-03-11T19:37:49.831 INFO:teuthology.orchestra.run.smithi107.stdout:(7/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 5.1 MB/s | 1.5 MB 00:00 2024-03-11T19:37:50.079 INFO:teuthology.orchestra.run.smithi122.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 4.6 MB/s | 2.1 MB 00:00 2024-03-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi122.stdout:(5/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 4.3 MB/s | 5.3 MB 00:01 2024-03-11T19:37:50.488 INFO:teuthology.orchestra.run.smithi122.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-03-11T19:37:50.840 INFO:teuthology.orchestra.run.smithi107.stdout:(8/174): ceph-radosgw-19.0.0-1607.g94a53970.el9 9.7 MB/s | 9.8 MB 00:01 2024-03-11T19:37:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 306 kB/s | 25 kB 00:00 2024-03-11T19:37:51.180 INFO:teuthology.orchestra.run.smithi122.stdout:(7/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-03-11T19:37:52.646 INFO:teuthology.orchestra.run.smithi122.stdout:(8/174): ceph-common-19.0.0-1607.g94a53970.el9. 5.6 MB/s | 20 MB 00:03 2024-03-11T19:37:52.722 INFO:teuthology.orchestra.run.smithi122.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 336 kB/s | 25 kB 00:00 2024-03-11T19:37:53.347 INFO:teuthology.orchestra.run.smithi122.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 4.5 MB/s | 9.8 MB 00:02 2024-03-11T19:37:53.514 INFO:teuthology.orchestra.run.smithi122.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.3 MB/s | 217 kB 00:00 2024-03-11T19:37:53.597 INFO:teuthology.orchestra.run.smithi122.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 324 kB/s | 27 kB 00:00 2024-03-11T19:37:53.673 INFO:teuthology.orchestra.run.smithi122.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 416 kB/s | 31 kB 00:00 2024-03-11T19:37:54.015 INFO:teuthology.orchestra.run.smithi122.stdout:(14/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 4.8 MB/s | 17 MB 00:03 2024-03-11T19:37:54.040 INFO:teuthology.orchestra.run.smithi122.stdout:(15/174): libcephfs2-19.0.0-1607.g94a53970.el9. 2.0 MB/s | 742 kB 00:00 2024-03-11T19:37:54.099 INFO:teuthology.orchestra.run.smithi122.stdout:(16/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.9 MB/s | 163 kB 00:00 2024-03-11T19:37:54.124 INFO:teuthology.orchestra.run.smithi122.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 1.5 MB/s | 127 kB 00:00 2024-03-11T19:37:54.257 INFO:teuthology.orchestra.run.smithi122.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 3.1 MB/s | 498 kB 00:00 2024-03-11T19:37:54.341 INFO:teuthology.orchestra.run.smithi122.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 541 kB/s | 45 kB 00:00 2024-03-11T19:37:54.425 INFO:teuthology.orchestra.run.smithi122.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.5 MB/s | 131 kB 00:00 2024-03-11T19:37:54.525 INFO:teuthology.orchestra.run.smithi122.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.6 MB/s | 161 kB 00:00 2024-03-11T19:37:54.584 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 849 kB/s | 19 MB 00:22 2024-03-11T19:37:54.625 INFO:teuthology.orchestra.run.smithi122.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 3.1 MB/s | 320 kB 00:00 2024-03-11T19:37:54.725 INFO:teuthology.orchestra.run.smithi122.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 2.9 MB/s | 298 kB 00:00 2024-03-11T19:37:54.784 INFO:teuthology.orchestra.run.smithi122.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.7 MB/s | 99 kB 00:00 2024-03-11T19:37:54.843 INFO:teuthology.orchestra.run.smithi122.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-11T19:37:55.026 INFO:teuthology.orchestra.run.smithi122.stdout:(26/174): librgw2-19.0.0-1607.g94a53970.el9.x86 5.7 MB/s | 5.1 MB 00:00 2024-03-11T19:37:55.093 INFO:teuthology.orchestra.run.smithi122.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-11T19:37:55.152 INFO:teuthology.orchestra.run.smithi122.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 539 kB/s | 31 kB 00:00 2024-03-11T19:37:55.227 INFO:teuthology.orchestra.run.smithi122.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.8 MB/s | 138 kB 00:00 2024-03-11T19:37:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:(10/174): ceph-test-19.0.0-1607.g94a53970.el9.x 11 MB/s | 47 MB 00:04 2024-03-11T19:37:55.278 INFO:teuthology.orchestra.run.smithi122.stdout:(30/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 7.1 MB/s | 3.1 MB 00:00 2024-03-11T19:37:55.465 INFO:teuthology.orchestra.run.smithi107.stdout:(11/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 2.9 MB/s | 17 MB 00:05 2024-03-11T19:37:55.498 INFO:teuthology.orchestra.run.smithi107.stdout:(12/174): cephfs-mirror-19.0.0-1607.g94a53970.e 820 kB/s | 217 kB 00:00 2024-03-11T19:37:55.503 INFO:teuthology.orchestra.run.smithi122.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 6.5 MB/s | 1.8 MB 00:00 2024-03-11T19:37:55.524 INFO:teuthology.orchestra.run.smithi107.stdout:(13/174): cephfs-shell-19.0.0-1607.g94a53970.el 469 kB/s | 27 kB 00:00 2024-03-11T19:37:55.549 INFO:teuthology.orchestra.run.smithi107.stdout:(14/174): libcephfs-devel-19.0.0-1607.g94a53970 629 kB/s | 31 kB 00:00 2024-03-11T19:37:55.570 INFO:teuthology.orchestra.run.smithi122.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 3.6 MB/s | 247 kB 00:00 2024-03-11T19:37:55.607 INFO:teuthology.orchestra.run.smithi107.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 2.8 MB/s | 163 kB 00:00 2024-03-11T19:37:55.628 INFO:teuthology.orchestra.run.smithi122.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 844 kB/s | 49 kB 00:00 2024-03-11T19:37:55.687 INFO:teuthology.orchestra.run.smithi122.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 263 kB/s | 15 kB 00:00 2024-03-11T19:37:55.706 INFO:teuthology.orchestra.run.smithi107.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.3 MB/s | 127 kB 00:00 2024-03-11T19:37:55.746 INFO:teuthology.orchestra.run.smithi122.stdout:(35/174): ceph-volume-19.0.0-1607.g94a53970.el9 4.4 MB/s | 264 kB 00:00 2024-03-11T19:37:55.791 INFO:teuthology.orchestra.run.smithi107.stdout:(17/174): libcephfs2-19.0.0-1607.g94a53970.el9. 2.7 MB/s | 742 kB 00:00 2024-03-11T19:37:55.850 INFO:teuthology.orchestra.run.smithi107.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 3.4 MB/s | 498 kB 00:00 2024-03-11T19:37:55.897 INFO:teuthology.orchestra.run.smithi122.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 4.9 MB/s | 748 kB 00:00 2024-03-11T19:37:55.900 INFO:teuthology.orchestra.run.smithi107.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 916 kB/s | 45 kB 00:00 2024-03-11T19:37:55.946 INFO:teuthology.orchestra.run.smithi122.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 364 kB/s | 18 kB 00:00 2024-03-11T19:37:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.9 MB/s | 131 kB 00:00 2024-03-11T19:37:56.034 INFO:teuthology.orchestra.run.smithi107.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 2.4 MB/s | 161 kB 00:00 2024-03-11T19:37:56.113 INFO:teuthology.orchestra.run.smithi122.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 662 kB 00:00 2024-03-11T19:37:56.117 INFO:teuthology.orchestra.run.smithi107.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 3.8 MB/s | 320 kB 00:00 2024-03-11T19:37:56.172 INFO:teuthology.orchestra.run.smithi122.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-11T19:37:56.193 INFO:teuthology.orchestra.run.smithi107.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.9 MB/s | 298 kB 00:00 2024-03-11T19:37:56.243 INFO:teuthology.orchestra.run.smithi107.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 2.0 MB/s | 99 kB 00:00 2024-03-11T19:37:56.264 INFO:teuthology.orchestra.run.smithi122.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-03-11T19:37:56.285 INFO:teuthology.orchestra.run.smithi107.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 2.0 MB/s | 85 kB 00:00 2024-03-11T19:37:56.289 INFO:teuthology.orchestra.run.smithi122.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 905 kB/s | 106 kB 00:00 2024-03-11T19:37:56.314 INFO:teuthology.orchestra.run.smithi122.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-11T19:37:56.331 INFO:teuthology.orchestra.run.smithi122.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 767 kB/s | 32 kB 00:00 2024-03-11T19:37:56.406 INFO:teuthology.orchestra.run.smithi122.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-11T19:37:56.423 INFO:teuthology.orchestra.run.smithi122.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 188 kB/s | 17 kB 00:00 2024-03-11T19:37:56.440 INFO:teuthology.orchestra.run.smithi122.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-11T19:37:56.466 INFO:teuthology.orchestra.run.smithi122.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-11T19:37:56.482 INFO:teuthology.orchestra.run.smithi122.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-11T19:37:56.516 INFO:teuthology.orchestra.run.smithi122.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-11T19:37:56.544 INFO:teuthology.orchestra.run.smithi107.stdout:(26/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 12 MB/s | 3.1 MB 00:00 2024-03-11T19:37:56.550 INFO:teuthology.orchestra.run.smithi122.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.4 MB/s | 578 kB 00:00 2024-03-11T19:37:56.567 INFO:teuthology.orchestra.run.smithi122.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 939 kB/s | 46 kB 00:00 2024-03-11T19:37:56.600 INFO:teuthology.orchestra.run.smithi122.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-11T19:37:56.625 INFO:teuthology.orchestra.run.smithi122.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 187 kB 00:00 2024-03-11T19:37:56.644 INFO:teuthology.orchestra.run.smithi107.stdout:(27/174): ceph-mon-19.0.0-1607.g94a53970.el9.x8 679 kB/s | 4.7 MB 00:07 2024-03-11T19:37:56.651 INFO:teuthology.orchestra.run.smithi122.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-11T19:37:56.676 INFO:teuthology.orchestra.run.smithi122.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-11T19:37:56.677 INFO:teuthology.orchestra.run.smithi107.stdout:(28/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.3 MB/s | 171 kB 00:00 2024-03-11T19:37:56.736 INFO:teuthology.orchestra.run.smithi107.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1607.g 344 kB/s | 31 kB 00:00 2024-03-11T19:37:56.761 INFO:teuthology.orchestra.run.smithi107.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 1.6 MB/s | 138 kB 00:00 2024-03-11T19:37:56.784 INFO:teuthology.orchestra.run.smithi122.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-11T19:37:56.819 INFO:teuthology.orchestra.run.smithi122.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-11T19:37:56.852 INFO:teuthology.orchestra.run.smithi122.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-11T19:37:56.862 INFO:teuthology.orchestra.run.smithi107.stdout:(31/174): librgw2-19.0.0-1607.g94a53970.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-03-11T19:37:56.878 INFO:teuthology.orchestra.run.smithi122.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-11T19:37:56.987 INFO:teuthology.orchestra.run.smithi107.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1607.g94 1.9 MB/s | 247 kB 00:00 2024-03-11T19:37:56.994 INFO:teuthology.orchestra.run.smithi122.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-11T19:37:57.036 INFO:teuthology.orchestra.run.smithi122.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-03-11T19:37:57.054 INFO:teuthology.orchestra.run.smithi107.stdout:(33/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 748 kB/s | 49 kB 00:00 2024-03-11T19:37:57.062 INFO:teuthology.orchestra.run.smithi122.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-11T19:37:57.121 INFO:teuthology.orchestra.run.smithi107.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1607.g9 232 kB/s | 15 kB 00:00 2024-03-11T19:37:57.157 INFO:teuthology.orchestra.run.smithi122.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 48 MB/s | 5.8 MB 00:00 2024-03-11T19:37:57.174 INFO:teuthology.orchestra.run.smithi122.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 223 kB/s | 24 kB 00:00 2024-03-11T19:37:57.191 INFO:teuthology.orchestra.run.smithi122.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 542 kB/s | 18 kB 00:00 2024-03-11T19:37:57.216 INFO:teuthology.orchestra.run.smithi122.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-11T19:37:57.233 INFO:teuthology.orchestra.run.smithi122.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 753 kB/s | 32 kB 00:00 2024-03-11T19:37:57.321 INFO:teuthology.orchestra.run.smithi107.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-11T19:37:57.325 INFO:teuthology.orchestra.run.smithi122.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 4.8 MB 00:00 2024-03-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi122.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 217 kB 00:00 2024-03-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi107.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.1 MB/s | 264 kB 00:00 2024-03-11T19:37:57.376 INFO:teuthology.orchestra.run.smithi122.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-11T19:37:57.401 INFO:teuthology.orchestra.run.smithi122.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-11T19:37:57.413 INFO:teuthology.orchestra.run.smithi107.stdout:(37/174): cephadm-19.0.0-1607.g94a53970.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-11T19:37:57.437 INFO:teuthology.orchestra.run.smithi122.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 704 kB/s | 41 kB 00:00 2024-03-11T19:37:57.438 INFO:teuthology.orchestra.run.smithi107.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 220 kB/s | 18 kB 00:00 2024-03-11T19:37:57.469 INFO:teuthology.orchestra.run.smithi122.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-11T19:37:57.494 INFO:teuthology.orchestra.run.smithi122.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-11T19:37:57.519 INFO:teuthology.orchestra.run.smithi122.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-11T19:37:57.544 INFO:teuthology.orchestra.run.smithi122.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-11T19:37:57.570 INFO:teuthology.orchestra.run.smithi122.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-11T19:37:57.670 INFO:teuthology.orchestra.run.smithi122.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 397 kB/s | 39 kB 00:00 2024-03-11T19:37:57.690 INFO:teuthology.orchestra.run.smithi107.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-11T19:37:57.731 INFO:teuthology.orchestra.run.smithi107.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-03-11T19:37:57.756 INFO:teuthology.orchestra.run.smithi107.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-03-11T19:37:57.789 INFO:teuthology.orchestra.run.smithi107.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-11T19:37:57.815 INFO:teuthology.orchestra.run.smithi107.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-03-11T19:37:57.845 INFO:teuthology.orchestra.run.smithi122.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 52 MB/s | 16 MB 00:00 2024-03-11T19:37:57.870 INFO:teuthology.orchestra.run.smithi122.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 208 kB/s | 41 kB 00:00 2024-03-11T19:37:57.896 INFO:teuthology.orchestra.run.smithi122.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-11T19:37:57.921 INFO:teuthology.orchestra.run.smithi122.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-11T19:37:57.940 INFO:teuthology.orchestra.run.smithi107.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-11T19:37:57.946 INFO:teuthology.orchestra.run.smithi122.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-11T19:37:57.965 INFO:teuthology.orchestra.run.smithi107.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-03-11T19:37:57.972 INFO:teuthology.orchestra.run.smithi122.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-11T19:37:57.997 INFO:teuthology.orchestra.run.smithi122.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-11T19:37:58.007 INFO:teuthology.orchestra.run.smithi107.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 248 kB/s | 10 kB 00:00 2024-03-11T19:37:58.022 INFO:teuthology.orchestra.run.smithi122.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-11T19:37:58.032 INFO:teuthology.orchestra.run.smithi107.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-03-11T19:37:58.056 INFO:teuthology.orchestra.run.smithi122.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-11T19:37:58.081 INFO:teuthology.orchestra.run.smithi122.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-11T19:37:58.082 INFO:teuthology.orchestra.run.smithi107.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-11T19:37:58.107 INFO:teuthology.orchestra.run.smithi122.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-11T19:37:58.124 INFO:teuthology.orchestra.run.smithi107.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-03-11T19:37:58.132 INFO:teuthology.orchestra.run.smithi122.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-11T19:37:58.166 INFO:teuthology.orchestra.run.smithi107.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-11T19:37:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-11T19:37:58.233 INFO:teuthology.orchestra.run.smithi107.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.5 MB/s | 282 kB 00:00 2024-03-11T19:37:58.292 INFO:teuthology.orchestra.run.smithi107.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 187 kB 00:00 2024-03-11T19:37:58.326 INFO:teuthology.orchestra.run.smithi107.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-03-11T19:37:58.351 INFO:teuthology.orchestra.run.smithi107.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2024-03-11T19:37:58.485 INFO:teuthology.orchestra.run.smithi107.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-11T19:37:58.527 INFO:teuthology.orchestra.run.smithi107.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.4 MB/s | 1.5 MB 00:00 2024-03-11T19:37:58.569 INFO:teuthology.orchestra.run.smithi107.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 9.6 MB/s | 811 kB 00:00 2024-03-11T19:37:58.594 INFO:teuthology.orchestra.run.smithi107.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-11T19:37:58.736 INFO:teuthology.orchestra.run.smithi107.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-11T19:37:58.763 INFO:teuthology.orchestra.run.smithi122.stdout:(91/174): ceph-test-19.0.0-1607.g94a53970.el9.x 7.8 MB/s | 47 MB 00:06 2024-03-11T19:37:58.778 INFO:teuthology.orchestra.run.smithi107.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-03-11T19:37:58.828 INFO:teuthology.orchestra.run.smithi107.stdout:(62/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 872 kB/s | 1.8 MB 00:02 2024-03-11T19:37:58.870 INFO:teuthology.orchestra.run.smithi107.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 855 kB/s | 78 kB 00:00 2024-03-11T19:37:58.883 INFO:teuthology.orchestra.run.smithi122.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 241 kB/s | 187 kB 00:00 2024-03-11T19:37:58.941 INFO:teuthology.orchestra.run.smithi122.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-11T19:37:59.029 INFO:teuthology.orchestra.run.smithi107.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-11T19:37:59.045 INFO:teuthology.orchestra.run.smithi107.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 113 kB/s | 24 kB 00:00 2024-03-11T19:37:59.062 INFO:teuthology.orchestra.run.smithi107.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 101 kB/s | 18 kB 00:00 2024-03-11T19:37:59.083 INFO:teuthology.orchestra.run.smithi122.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 844 kB/s | 803 kB 00:00 2024-03-11T19:37:59.104 INFO:teuthology.orchestra.run.smithi107.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 788 kB/s | 32 kB 00:00 2024-03-11T19:37:59.146 INFO:teuthology.orchestra.run.smithi107.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-11T19:37:59.263 INFO:teuthology.orchestra.run.smithi107.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-03-11T19:37:59.275 INFO:teuthology.orchestra.run.smithi122.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 64 kB/s | 21 kB 00:00 2024-03-11T19:37:59.288 INFO:teuthology.orchestra.run.smithi107.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-11T19:37:59.313 INFO:teuthology.orchestra.run.smithi107.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-11T19:37:59.375 INFO:teuthology.orchestra.run.smithi122.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 96 kB/s | 25 kB 00:00 2024-03-11T19:37:59.389 INFO:teuthology.orchestra.run.smithi107.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 933 kB/s | 69 kB 00:00 2024-03-11T19:37:59.405 INFO:teuthology.orchestra.run.smithi107.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 456 kB/s | 41 kB 00:00 2024-03-11T19:37:59.430 INFO:teuthology.orchestra.run.smithi107.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 238 kB 00:00 2024-03-11T19:37:59.481 INFO:teuthology.orchestra.run.smithi107.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2024-03-11T19:37:59.506 INFO:teuthology.orchestra.run.smithi107.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-11T19:37:59.523 INFO:teuthology.orchestra.run.smithi107.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 524 kB/s | 47 kB 00:00 2024-03-11T19:37:59.556 INFO:teuthology.orchestra.run.smithi107.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-11T19:37:59.581 INFO:teuthology.orchestra.run.smithi107.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 680 kB/s | 39 kB 00:00 2024-03-11T19:37:59.592 INFO:teuthology.orchestra.run.smithi122.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-11T19:37:59.709 INFO:teuthology.orchestra.run.smithi122.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.5 MB/s | 4.1 MB 00:00 2024-03-11T19:37:59.743 INFO:teuthology.orchestra.run.smithi122.stdout:(99/174): libquadmath-11.4.1-3.el9.x86_64.rpm 199 kB/s | 195 kB 00:00 2024-03-11T19:37:59.768 INFO:teuthology.orchestra.run.smithi122.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.4 MB/s | 248 kB 00:00 2024-03-11T19:37:59.865 INFO:teuthology.orchestra.run.smithi107.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-03-11T19:37:59.893 INFO:teuthology.orchestra.run.smithi122.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-11T19:37:59.898 INFO:teuthology.orchestra.run.smithi107.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 989 kB/s | 300 kB 00:00 2024-03-11T19:37:59.919 INFO:teuthology.orchestra.run.smithi122.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 296 kB/s | 44 kB 00:00 2024-03-11T19:37:59.924 INFO:teuthology.orchestra.run.smithi107.stdout:(82/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 124 kB/s | 41 kB 00:00 2024-03-11T19:37:59.949 INFO:teuthology.orchestra.run.smithi107.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 702 kB/s | 56 kB 00:00 2024-03-11T19:37:59.960 INFO:teuthology.orchestra.run.smithi122.stdout:(103/174): qatlib-service-23.11.0-1.el9.x86_64. 160 kB/s | 35 kB 00:00 2024-03-11T19:37:59.986 INFO:teuthology.orchestra.run.smithi122.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-03-11T19:37:59.991 INFO:teuthology.orchestra.run.smithi107.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 690 kB/s | 63 kB 00:00 2024-03-11T19:38:00.011 INFO:teuthology.orchestra.run.smithi122.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 800 kB/s | 94 kB 00:00 2024-03-11T19:38:00.016 INFO:teuthology.orchestra.run.smithi107.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 674 kB/s | 44 kB 00:00 2024-03-11T19:38:00.033 INFO:teuthology.orchestra.run.smithi107.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 960 kB/s | 79 kB 00:00 2024-03-11T19:38:00.067 INFO:teuthology.orchestra.run.smithi107.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-11T19:38:00.092 INFO:teuthology.orchestra.run.smithi107.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi122.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi107.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 9.4 MB/s | 1.1 MB 00:00 2024-03-11T19:38:00.184 INFO:teuthology.orchestra.run.smithi107.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-11T19:38:00.218 INFO:teuthology.orchestra.run.smithi107.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 187 kB 00:00 2024-03-11T19:38:00.276 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.1 MB 00:02 2024-03-11T19:38:00.495 INFO:teuthology.orchestra.run.smithi122.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-03-11T19:38:00.687 INFO:teuthology.orchestra.run.smithi122.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 457 kB/s | 309 kB 00:00 2024-03-11T19:38:00.854 INFO:teuthology.orchestra.run.smithi122.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-03-11T19:38:00.879 INFO:teuthology.orchestra.run.smithi122.stdout:(110/174): abseil-cpp-20211102.0-3.el9.x86_64.r 614 kB/s | 548 kB 00:00 2024-03-11T19:38:00.996 INFO:teuthology.orchestra.run.smithi122.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-03-11T19:38:01.021 INFO:teuthology.orchestra.run.smithi122.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-03-11T19:38:01.163 INFO:teuthology.orchestra.run.smithi122.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 905 kB/s | 151 kB 00:00 2024-03-11T19:38:01.272 INFO:teuthology.orchestra.run.smithi122.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-03-11T19:38:01.414 INFO:teuthology.orchestra.run.smithi122.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 306 kB/s | 43 kB 00:00 2024-03-11T19:38:01.557 INFO:teuthology.orchestra.run.smithi122.stdout:(116/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-03-11T19:38:01.582 INFO:teuthology.orchestra.run.smithi122.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 192 kB/s | 32 kB 00:00 2024-03-11T19:38:01.607 INFO:teuthology.orchestra.run.smithi122.stdout:(118/174): python3-asyncssh-2.13.2-4.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-03-11T19:38:01.634 INFO:teuthology.orchestra.run.smithi107.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 129 kB/s | 187 kB 00:01 2024-03-11T19:38:01.707 INFO:teuthology.orchestra.run.smithi122.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 100 kB/s | 15 kB 00:00 2024-03-11T19:38:01.785 INFO:teuthology.orchestra.run.smithi107.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 125 kB/s | 195 kB 00:01 2024-03-11T19:38:01.941 INFO:teuthology.orchestra.run.smithi122.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-03-11T19:38:02.066 INFO:teuthology.orchestra.run.smithi122.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-03-11T19:38:02.101 INFO:teuthology.orchestra.run.smithi107.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 68 kB/s | 21 kB 00:00 2024-03-11T19:38:02.227 INFO:teuthology.orchestra.run.smithi107.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-03-11T19:38:02.250 INFO:teuthology.orchestra.run.smithi122.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.1 MB/s | 203 kB 00:00 2024-03-11T19:38:02.275 INFO:teuthology.orchestra.run.smithi122.stdout:(123/174): python3-cheroot-8.6.0-4.el9.noarch.r 248 kB/s | 172 kB 00:00 2024-03-11T19:38:02.400 INFO:teuthology.orchestra.run.smithi122.stdout:(124/174): python3-cherrypy-18.6.1-2.el9.noarch 452 kB/s | 358 kB 00:00 2024-03-11T19:38:02.425 INFO:teuthology.orchestra.run.smithi122.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 962 kB/s | 144 kB 00:00 2024-03-11T19:38:02.526 INFO:teuthology.orchestra.run.smithi122.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-03-11T19:38:02.568 INFO:teuthology.orchestra.run.smithi122.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 124 kB/s | 18 kB 00:00 2024-03-11T19:38:02.610 INFO:teuthology.orchestra.run.smithi122.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-03-11T19:38:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-03-11T19:38:02.718 INFO:teuthology.orchestra.run.smithi122.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-11T19:38:02.735 INFO:teuthology.orchestra.run.smithi122.stdout:(131/174): python3-jaraco-functools-3.5.0-2.el9 116 kB/s | 19 kB 00:00 2024-03-11T19:38:02.765 INFO:teuthology.orchestra.run.smithi107.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 311 kB/s | 803 kB 00:02 2024-03-11T19:38:02.803 INFO:teuthology.orchestra.run.smithi107.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 107 kB/s | 125 kB 00:01 2024-03-11T19:38:02.818 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 15 kB 00:01 2024-03-11T19:38:02.852 INFO:teuthology.orchestra.run.smithi122.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-03-11T19:38:02.877 INFO:teuthology.orchestra.run.smithi122.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 556 kB/s | 79 kB 00:00 2024-03-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi122.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 495 kB/s | 58 kB 00:00 2024-03-11T19:38:03.045 INFO:teuthology.orchestra.run.smithi122.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-11T19:38:03.087 INFO:teuthology.orchestra.run.smithi122.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-03-11T19:38:03.103 INFO:teuthology.orchestra.run.smithi122.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 122 kB/s | 16 kB 00:00 2024-03-11T19:38:03.162 INFO:teuthology.orchestra.run.smithi122.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-03-11T19:38:03.237 INFO:teuthology.orchestra.run.smithi122.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 188 kB/s | 28 kB 00:00 2024-03-11T19:38:03.262 INFO:teuthology.orchestra.run.smithi122.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2024-03-11T19:38:03.278 INFO:teuthology.orchestra.run.smithi107.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.9 MB/s | 4.1 MB 00:01 2024-03-11T19:38:03.313 INFO:teuthology.orchestra.run.smithi122.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-11T19:38:03.355 INFO:teuthology.orchestra.run.smithi122.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-03-11T19:38:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-03-11T19:38:03.394 INFO:teuthology.orchestra.run.smithi107.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.8 MB/s | 216 kB 00:00 2024-03-11T19:38:03.439 INFO:teuthology.orchestra.run.smithi122.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-03-11T19:38:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-03-11T19:38:03.520 INFO:teuthology.orchestra.run.smithi107.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 280 kB/s | 35 kB 00:00 2024-03-11T19:38:03.539 INFO:teuthology.orchestra.run.smithi122.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 565 kB/s | 90 kB 00:00 2024-03-11T19:38:03.600 INFO:teuthology.orchestra.run.smithi122.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-03-11T19:38:03.645 INFO:teuthology.orchestra.run.smithi107.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 357 kB/s | 44 kB 00:00 2024-03-11T19:38:03.650 INFO:teuthology.orchestra.run.smithi122.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 147 kB/s | 20 kB 00:00 2024-03-11T19:38:03.675 INFO:teuthology.orchestra.run.smithi122.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-11T19:38:03.779 INFO:teuthology.orchestra.run.smithi107.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 706 kB/s | 94 kB 00:00 2024-03-11T19:38:03.834 INFO:teuthology.orchestra.run.smithi122.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-11T19:38:03.984 INFO:teuthology.orchestra.run.smithi122.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2024-03-11T19:38:03.995 INFO:teuthology.orchestra.run.smithi107.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-03-11T19:38:04.034 INFO:teuthology.orchestra.run.smithi122.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 8.8 MB/s | 3.4 MB 00:00 2024-03-11T19:38:04.118 INFO:teuthology.orchestra.run.smithi122.stdout:(153/174): librbd1-19.0.0-1607.g94a53970.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-03-11T19:38:04.360 INFO:teuthology.orchestra.run.smithi122.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2024-03-11T19:38:04.396 INFO:teuthology.orchestra.run.smithi107.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-03-11T19:38:04.419 INFO:teuthology.orchestra.run.smithi122.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-11T19:38:04.429 INFO:teuthology.orchestra.run.smithi107.stdout:(105/174): python3-devel-3.9.17-2.el9.x86_64.rp 148 kB/s | 245 kB 00:01 2024-03-11T19:38:04.454 INFO:teuthology.orchestra.run.smithi107.stdout:(106/174): python3-jinja2-2.11.3-5.el9.noarch.r 150 kB/s | 248 kB 00:01 2024-03-11T19:38:04.477 INFO:teuthology.orchestra.run.smithi122.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-11T19:38:04.528 INFO:teuthology.orchestra.run.smithi122.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.7 MB/s | 1.8 MB 00:00 2024-03-11T19:38:04.546 INFO:teuthology.orchestra.run.smithi107.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-03-11T19:38:04.553 INFO:teuthology.orchestra.run.smithi122.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 483 kB/s | 36 kB 00:00 2024-03-11T19:38:04.586 INFO:teuthology.orchestra.run.smithi122.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-11T19:38:04.645 INFO:teuthology.orchestra.run.smithi122.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 397 kB/s | 23 kB 00:00 2024-03-11T19:38:04.663 INFO:teuthology.orchestra.run.smithi107.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-03-11T19:38:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-11T19:38:04.697 INFO:teuthology.orchestra.run.smithi122.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-11T19:38:04.705 INFO:teuthology.orchestra.run.smithi107.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-03-11T19:38:04.731 INFO:teuthology.orchestra.run.smithi107.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 393 kB/s | 26 kB 00:00 2024-03-11T19:38:04.754 INFO:teuthology.orchestra.run.smithi122.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-11T19:38:04.772 INFO:teuthology.orchestra.run.smithi107.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 736 kB/s | 49 kB 00:00 2024-03-11T19:38:04.779 INFO:teuthology.orchestra.run.smithi122.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-11T19:38:04.798 INFO:teuthology.orchestra.run.smithi107.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-11T19:38:04.804 INFO:teuthology.orchestra.run.smithi122.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-11T19:38:04.830 INFO:teuthology.orchestra.run.smithi122.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-11T19:38:04.831 INFO:teuthology.orchestra.run.smithi107.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-03-11T19:38:04.964 INFO:teuthology.orchestra.run.smithi122.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-03-11T19:38:04.990 INFO:teuthology.orchestra.run.smithi107.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-11T19:38:05.040 INFO:teuthology.orchestra.run.smithi107.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-03-11T19:38:05.065 INFO:teuthology.orchestra.run.smithi107.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 585 kB/s | 43 kB 00:00 2024-03-11T19:38:05.091 INFO:teuthology.orchestra.run.smithi107.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 653 kB/s | 32 kB 00:00 2024-03-11T19:38:05.124 INFO:teuthology.orchestra.run.smithi107.stdout:(118/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-11T19:38:05.149 INFO:teuthology.orchestra.run.smithi107.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-03-11T19:38:05.183 INFO:teuthology.orchestra.run.smithi107.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-11T19:38:05.216 INFO:teuthology.orchestra.run.smithi107.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-03-11T19:38:05.250 INFO:teuthology.orchestra.run.smithi107.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-11T19:38:05.267 INFO:teuthology.orchestra.run.smithi107.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 426 kB/s | 35 kB 00:00 2024-03-11T19:38:05.300 INFO:teuthology.orchestra.run.smithi107.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-11T19:38:05.351 INFO:teuthology.orchestra.run.smithi107.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-11T19:38:05.368 INFO:teuthology.orchestra.run.smithi107.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-11T19:38:05.393 INFO:teuthology.orchestra.run.smithi107.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 428 kB/s | 18 kB 00:00 2024-03-11T19:38:05.418 INFO:teuthology.orchestra.run.smithi107.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 467 kB/s | 23 kB 00:00 2024-03-11T19:38:05.444 INFO:teuthology.orchestra.run.smithi107.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-03-11T19:38:05.469 INFO:teuthology.orchestra.run.smithi107.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2024-03-11T19:38:05.544 INFO:teuthology.orchestra.run.smithi107.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-03-11T19:38:05.570 INFO:teuthology.orchestra.run.smithi107.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 466 kB/s | 46 kB 00:00 2024-03-11T19:38:05.612 INFO:teuthology.orchestra.run.smithi107.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-03-11T19:38:05.637 INFO:teuthology.orchestra.run.smithi107.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 871 kB/s | 58 kB 00:00 2024-03-11T19:38:05.654 INFO:teuthology.orchestra.run.smithi107.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 728 kB/s | 79 kB 00:00 2024-03-11T19:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-11T19:38:05.746 INFO:teuthology.orchestra.run.smithi107.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-11T19:38:05.771 INFO:teuthology.orchestra.run.smithi107.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 495 kB/s | 28 kB 00:00 2024-03-11T19:38:05.796 INFO:teuthology.orchestra.run.smithi107.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 634 kB/s | 90 kB 00:00 2024-03-11T19:38:05.797 INFO:teuthology.orchestra.run.smithi122.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 2.3 MB 00:00 2024-03-11T19:38:05.822 INFO:teuthology.orchestra.run.smithi107.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-03-11T19:38:05.863 INFO:teuthology.orchestra.run.smithi107.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi107.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 487 kB/s | 36 kB 00:00 2024-03-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi122.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2024-03-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi107.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 598 kB/s | 59 kB 00:00 2024-03-11T19:38:05.947 INFO:teuthology.orchestra.run.smithi107.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 612 kB/s | 51 kB 00:00 2024-03-11T19:38:05.956 INFO:teuthology.orchestra.run.smithi122.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 749 kB/s | 31 kB 00:00 2024-03-11T19:38:05.973 INFO:teuthology.orchestra.run.smithi107.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-03-11T19:38:06.006 INFO:teuthology.orchestra.run.smithi107.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-11T19:38:06.023 INFO:teuthology.orchestra.run.smithi122.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.0 MB/s | 465 kB 00:00 2024-03-11T19:38:06.031 INFO:teuthology.orchestra.run.smithi107.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-03-11T19:38:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-03-11T19:38:06.149 INFO:teuthology.orchestra.run.smithi107.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-03-11T19:38:06.216 INFO:teuthology.orchestra.run.smithi107.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-03-11T19:38:06.266 INFO:teuthology.orchestra.run.smithi107.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-11T19:38:06.273 INFO:teuthology.orchestra.run.smithi122.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 297 kB/s | 74 kB 00:00 2024-03-11T19:38:06.339 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 21 MB 00:03 2024-03-11T19:38:06.415 INFO:teuthology.orchestra.run.smithi122.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.3 MB/s | 1.1 MB 00:00 2024-03-11T19:38:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:(152/174): librbd1-19.0.0-1607.g94a53970.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-03-11T19:38:06.648 INFO:teuthology.orchestra.run.smithi122.stdout:(174/174): NetworkManager-team-1.46.0-1.el9.x86 17 kB/s | 42 kB 00:02 2024-03-11T19:38:06.653 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:38:06.653 INFO:teuthology.orchestra.run.smithi122.stdout:Total 12 MB/s | 223 MB 00:18 2024-03-11T19:38:07.137 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-03-11T19:38:07.409 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-03-11T19:38:07.409 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-03-11T19:38:07.565 INFO:teuthology.orchestra.run.smithi107.stdout:(153/174): librados2-19.0.0-1607.g94a53970.el9. 2.3 MB/s | 3.4 MB 00:01 2024-03-11T19:38:08.325 INFO:teuthology.orchestra.run.smithi107.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 56 kB/s | 42 kB 00:00 2024-03-11T19:38:09.114 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-03-11T19:38:09.114 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-03-11T19:38:09.425 INFO:teuthology.orchestra.run.smithi107.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 226 kB/s | 248 kB 00:01 2024-03-11T19:38:10.075 INFO:teuthology.orchestra.run.smithi107.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 551 kB/s | 1.8 MB 00:03 2024-03-11T19:38:10.225 INFO:teuthology.orchestra.run.smithi107.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 89 kB/s | 71 kB 00:00 2024-03-11T19:38:10.451 INFO:teuthology.orchestra.run.smithi107.stdout:(158/174): NetworkManager-1.46.0-1.el9.x86_64.r 560 kB/s | 2.3 MB 00:04 2024-03-11T19:38:10.892 INFO:teuthology.orchestra.run.smithi107.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 45 kB/s | 36 kB 00:00 2024-03-11T19:38:11.259 INFO:teuthology.orchestra.run.smithi107.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 186 kB/s | 192 kB 00:01 2024-03-11T19:38:11.394 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-11T19:38:11.626 INFO:teuthology.orchestra.run.smithi107.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 218 kB/s | 256 kB 00:01 2024-03-11T19:38:11.643 INFO:teuthology.orchestra.run.smithi107.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 31 kB/s | 23 kB 00:00 2024-03-11T19:38:12.227 INFO:teuthology.orchestra.run.smithi107.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 112 kB/s | 108 kB 00:00 2024-03-11T19:38:12.593 INFO:teuthology.orchestra.run.smithi107.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 143 kB/s | 136 kB 00:00 2024-03-11T19:38:12.627 INFO:teuthology.orchestra.run.smithi107.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 155 kB/s | 155 kB 00:00 2024-03-11T19:38:13.094 INFO:teuthology.orchestra.run.smithi107.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 74 kB/s | 64 kB 00:00 2024-03-11T19:38:13.336 INFO:teuthology.orchestra.run.smithi107.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 39 kB/s | 29 kB 00:00 2024-03-11T19:38:13.841 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:38:13.842 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:38:13.856 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:38:13.857 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:38:13.858 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:38:13.859 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:38:13.862 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:38:13.864 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:38:13.865 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:38:13.944 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-03-11T19:38:13.963 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-03-11T19:38:13.963 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-03-11T19:38:13.963 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-03-11T19:38:13.963 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-03-11T19:38:13.963 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:38:13.964 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:38:13.965 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:38:13.966 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:38:13.967 INFO:teuthology.orchestra.run.smithi145.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:38:13.968 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:38:13.969 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:38:13.970 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:38:13.971 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:38:13.972 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:38:13.973 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:38:13.974 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:38:13.975 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================================ 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout:Install 150 Packages 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 24 Packages 2024-03-11T19:38:13.976 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:38:13.977 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 223 M 2024-03-11T19:38:13.977 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-03-11T19:38:15.020 INFO:teuthology.orchestra.run.smithi145.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-03-11T19:38:15.169 INFO:teuthology.orchestra.run.smithi107.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 488 kB/s | 1.2 MB 00:02 2024-03-11T19:38:15.287 INFO:teuthology.orchestra.run.smithi145.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 3.2 MB/s | 886 kB 00:00 2024-03-11T19:38:15.404 INFO:teuthology.orchestra.run.smithi145.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.2 MB/s | 145 kB 00:00 2024-03-11T19:38:15.663 INFO:teuthology.orchestra.run.smithi145.stdout:(4/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 6.3 MB/s | 5.3 MB 00:00 2024-03-11T19:38:15.813 INFO:teuthology.orchestra.run.smithi145.stdout:(5/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 5.2 MB/s | 2.1 MB 00:00 2024-03-11T19:38:16.049 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-03-11T19:38:16.178 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:38:16.212 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:38:16.243 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:16.246 INFO:teuthology.orchestra.run.smithi145.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-03-11T19:38:16.252 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:16.292 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:38:16.345 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:38:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:38:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:38:16.438 INFO:teuthology.orchestra.run.smithi145.stdout:(7/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-03-11T19:38:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:38:16.537 INFO:teuthology.orchestra.run.smithi107.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 341 kB/s | 465 kB 00:01 2024-03-11T19:38:16.630 INFO:teuthology.orchestra.run.smithi145.stdout:(8/174): ceph-common-19.0.0-1607.g94a53970.el9. 11 MB/s | 20 MB 00:01 2024-03-11T19:38:16.706 INFO:teuthology.orchestra.run.smithi145.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 336 kB/s | 25 kB 00:00 2024-03-11T19:38:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:38:16.831 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:16.912 INFO:teuthology.orchestra.run.smithi107.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 579 kB/s | 2.2 MB 00:03 2024-03-11T19:38:16.944 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:17.078 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:38:17.117 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:38:17.253 INFO:teuthology.orchestra.run.smithi107.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 43 kB/s | 31 kB 00:00 2024-03-11T19:38:17.280 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:38:17.312 INFO:teuthology.orchestra.run.smithi107.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 588 kB/s | 2.3 MB 00:03 2024-03-11T19:38:17.326 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:38:17.354 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:17.441 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:17.463 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:38:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:17.679 INFO:teuthology.orchestra.run.smithi107.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 175 kB/s | 74 kB 00:00 2024-03-11T19:38:17.722 INFO:teuthology.orchestra.run.smithi145.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 7.6 MB/s | 9.8 MB 00:01 2024-03-11T19:38:17.723 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:17.754 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:17.794 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:38:17.831 INFO:teuthology.orchestra.run.smithi145.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 2.0 MB/s | 217 kB 00:00 2024-03-11T19:38:17.901 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:38:17.914 INFO:teuthology.orchestra.run.smithi145.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 325 kB/s | 27 kB 00:00 2024-03-11T19:38:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:18.023 INFO:teuthology.orchestra.run.smithi145.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 288 kB/s | 31 kB 00:00 2024-03-11T19:38:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:18.105 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:18.166 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:18.215 INFO:teuthology.orchestra.run.smithi145.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 3.8 MB/s | 742 kB 00:00 2024-03-11T19:38:18.316 INFO:teuthology.orchestra.run.smithi145.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.6 MB/s | 163 kB 00:00 2024-03-11T19:38:18.353 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:18.408 INFO:teuthology.orchestra.run.smithi145.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.4 MB/s | 127 kB 00:00 2024-03-11T19:38:18.463 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:38:18.516 INFO:teuthology.orchestra.run.smithi145.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 4.5 MB/s | 498 kB 00:00 2024-03-11T19:38:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:38:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:38:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:38:18.658 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:38:18.731 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:38:18.777 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:38:18.783 INFO:teuthology.orchestra.run.smithi145.stdout:(18/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 6.6 MB/s | 17 MB 00:02 2024-03-11T19:38:18.818 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:38:18.879 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:38:18.883 INFO:teuthology.orchestra.run.smithi145.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 453 kB/s | 45 kB 00:00 2024-03-11T19:38:18.915 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:38:18.958 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:38:18.984 INFO:teuthology.orchestra.run.smithi145.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.3 MB/s | 131 kB 00:00 2024-03-11T19:38:18.991 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:38:19.028 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:38:19.084 INFO:teuthology.orchestra.run.smithi145.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.6 MB/s | 161 kB 00:00 2024-03-11T19:38:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:38:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:38:19.175 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:38:19.187 INFO:teuthology.orchestra.run.smithi107.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 483 kB/s | 1.1 MB 00:02 2024-03-11T19:38:19.195 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:38:19.195 INFO:teuthology.orchestra.run.smithi107.stdout:Total 7.0 MB/s | 223 MB 00:31 2024-03-11T19:38:19.201 INFO:teuthology.orchestra.run.smithi145.stdout:(22/174): librgw2-19.0.0-1607.g94a53970.el9.x86 7.5 MB/s | 5.1 MB 00:00 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:38:19.226 INFO:teuthology.orchestra.run.smithi145.stdout:(23/174): python3-rados-19.0.0-1607.g94a53970.e 2.2 MB/s | 320 kB 00:00 2024-03-11T19:38:19.272 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:38:19.285 INFO:teuthology.orchestra.run.smithi145.stdout:(24/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.5 MB/s | 298 kB 00:00 2024-03-11T19:38:19.310 INFO:teuthology.orchestra.run.smithi145.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.2 MB/s | 99 kB 00:00 2024-03-11T19:38:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:38:19.360 INFO:teuthology.orchestra.run.smithi145.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-11T19:38:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:38:19.453 INFO:teuthology.orchestra.run.smithi145.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 1.8 MB/s | 171 kB 00:00 2024-03-11T19:38:19.536 INFO:teuthology.orchestra.run.smithi145.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 376 kB/s | 31 kB 00:00 2024-03-11T19:38:19.541 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:38:19.596 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:38:19.995 INFO:teuthology.orchestra.run.smithi145.stdout:(29/174): ceph-test-19.0.0-1607.g94a53970.el9.x 14 MB/s | 47 MB 00:03 2024-03-11T19:38:20.020 INFO:teuthology.orchestra.run.smithi145.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 285 kB/s | 138 kB 00:00 2024-03-11T19:38:20.087 INFO:teuthology.orchestra.run.smithi145.stdout:(31/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 4.0 MB/s | 3.1 MB 00:00 2024-03-11T19:38:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:38:20.171 INFO:teuthology.orchestra.run.smithi145.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 10 MB/s | 1.8 MB 00:00 2024-03-11T19:38:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:38:20.204 INFO:teuthology.orchestra.run.smithi145.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.1 MB/s | 247 kB 00:00 2024-03-11T19:38:20.246 INFO:teuthology.orchestra.run.smithi145.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 656 kB/s | 49 kB 00:00 2024-03-11T19:38:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:38:20.271 INFO:teuthology.orchestra.run.smithi145.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 229 kB/s | 15 kB 00:00 2024-03-11T19:38:20.313 INFO:teuthology.orchestra.run.smithi145.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 3.9 MB/s | 264 kB 00:00 2024-03-11T19:38:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:38:20.364 INFO:teuthology.orchestra.run.smithi145.stdout:(37/174): cephadm-19.0.0-1607.g94a53970.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-11T19:38:20.389 INFO:teuthology.orchestra.run.smithi145.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 242 kB/s | 18 kB 00:00 2024-03-11T19:38:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:38:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:38:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:38:20.572 INFO:teuthology.orchestra.run.smithi145.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-11T19:38:20.614 INFO:teuthology.orchestra.run.smithi145.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2024-03-11T19:38:20.622 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-11T19:38:20.640 INFO:teuthology.orchestra.run.smithi145.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-11T19:38:20.665 INFO:teuthology.orchestra.run.smithi145.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-11T19:38:20.682 INFO:teuthology.orchestra.run.smithi145.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 763 kB/s | 32 kB 00:00 2024-03-11T19:38:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:38:20.782 INFO:teuthology.orchestra.run.smithi145.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-03-11T19:38:20.807 INFO:teuthology.orchestra.run.smithi145.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 17 kB 00:00 2024-03-11T19:38:20.858 INFO:teuthology.orchestra.run.smithi145.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-11T19:38:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:38:20.883 INFO:teuthology.orchestra.run.smithi145.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2024-03-11T19:38:20.908 INFO:teuthology.orchestra.run.smithi145.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-03-11T19:38:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:38:20.933 INFO:teuthology.orchestra.run.smithi145.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-11T19:38:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:38:20.967 INFO:teuthology.orchestra.run.smithi145.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-11T19:38:20.992 INFO:teuthology.orchestra.run.smithi145.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-11T19:38:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-11T19:38:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-11T19:38:21.017 INFO:teuthology.orchestra.run.smithi145.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 46 kB 00:00 2024-03-11T19:38:21.051 INFO:teuthology.orchestra.run.smithi145.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-11T19:38:21.076 INFO:teuthology.orchestra.run.smithi145.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-11T19:38:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:38:21.102 INFO:teuthology.orchestra.run.smithi145.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-11T19:38:21.127 INFO:teuthology.orchestra.run.smithi145.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-11T19:38:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:38:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:38:21.219 INFO:teuthology.orchestra.run.smithi145.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-11T19:38:21.261 INFO:teuthology.orchestra.run.smithi145.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2024-03-11T19:38:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:38:21.286 INFO:teuthology.orchestra.run.smithi145.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.1 MB/s | 811 kB 00:00 2024-03-11T19:38:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:38:21.320 INFO:teuthology.orchestra.run.smithi145.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2024-03-11T19:38:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:38:21.353 INFO:teuthology.orchestra.run.smithi145.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-11T19:38:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:38:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:38:21.454 INFO:teuthology.orchestra.run.smithi145.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-11T19:38:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:38:21.470 INFO:teuthology.orchestra.run.smithi145.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 517 kB/s | 78 kB 00:00 2024-03-11T19:38:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:38:21.562 INFO:teuthology.orchestra.run.smithi145.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-11T19:38:21.588 INFO:teuthology.orchestra.run.smithi145.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 187 kB/s | 24 kB 00:00 2024-03-11T19:38:21.613 INFO:teuthology.orchestra.run.smithi145.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-11T19:38:21.655 INFO:teuthology.orchestra.run.smithi145.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 765 kB/s | 32 kB 00:00 2024-03-11T19:38:21.680 INFO:teuthology.orchestra.run.smithi145.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-11T19:38:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:38:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:38:21.756 INFO:teuthology.orchestra.run.smithi145.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-11T19:38:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:38:21.781 INFO:teuthology.orchestra.run.smithi145.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-11T19:38:21.806 INFO:teuthology.orchestra.run.smithi145.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-11T19:38:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:38:21.832 INFO:teuthology.orchestra.run.smithi145.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-11T19:38:21.849 INFO:teuthology.orchestra.run.smithi145.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-03-11T19:38:21.874 INFO:teuthology.orchestra.run.smithi145.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-11T19:38:21.899 INFO:teuthology.orchestra.run.smithi145.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-11T19:38:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:38:21.916 INFO:teuthology.orchestra.run.smithi145.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-11T19:38:21.942 INFO:teuthology.orchestra.run.smithi145.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-11T19:38:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:38:21.960 INFO:teuthology.orchestra.run.smithi145.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 43 kB 00:00 2024-03-11T19:38:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:38:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:38:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:38:22.094 INFO:teuthology.orchestra.run.smithi145.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 300 kB/s | 39 kB 00:00 2024-03-11T19:38:22.106 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:38:22.109 INFO:teuthology.orchestra.run.smithi145.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 282 kB/s | 41 kB 00:00 2024-03-11T19:38:22.236 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:38:22.259 INFO:teuthology.orchestra.run.smithi145.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-03-11T19:38:22.285 INFO:teuthology.orchestra.run.smithi145.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 324 kB/s | 56 kB 00:00 2024-03-11T19:38:22.310 INFO:teuthology.orchestra.run.smithi145.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-11T19:38:22.335 INFO:teuthology.orchestra.run.smithi145.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 833 kB/s | 63 kB 00:00 2024-03-11T19:38:22.337 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:38:22.360 INFO:teuthology.orchestra.run.smithi145.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2024-03-11T19:38:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:38:22.377 INFO:teuthology.orchestra.run.smithi145.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-11T19:38:22.402 INFO:teuthology.orchestra.run.smithi145.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-11T19:38:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:38:22.436 INFO:teuthology.orchestra.run.smithi145.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-11T19:38:22.461 INFO:teuthology.orchestra.run.smithi145.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-11T19:38:22.480 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:38:22.486 INFO:teuthology.orchestra.run.smithi145.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-11T19:38:22.512 INFO:teuthology.orchestra.run.smithi145.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-11T19:38:22.587 INFO:teuthology.orchestra.run.smithi145.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-03-11T19:38:22.612 INFO:teuthology.orchestra.run.smithi145.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-03-11T19:38:22.666 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:38:22.671 INFO:teuthology.orchestra.run.smithi145.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-11T19:38:22.696 INFO:teuthology.orchestra.run.smithi145.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-11T19:38:22.713 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:38:22.722 INFO:teuthology.orchestra.run.smithi145.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-03-11T19:38:22.763 INFO:teuthology.orchestra.run.smithi145.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-03-11T19:38:22.844 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:38:22.872 INFO:teuthology.orchestra.run.smithi145.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-11T19:38:22.887 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:38:22.919 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:38:22.922 INFO:teuthology.orchestra.run.smithi145.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-11T19:38:22.948 INFO:teuthology.orchestra.run.smithi145.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-03-11T19:38:22.950 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:38:22.973 INFO:teuthology.orchestra.run.smithi145.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 696 kB/s | 35 kB 00:00 2024-03-11T19:38:22.978 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:38:22.998 INFO:teuthology.orchestra.run.smithi145.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 890 kB/s | 44 kB 00:00 2024-03-11T19:38:23.012 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:38:23.023 INFO:teuthology.orchestra.run.smithi145.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-11T19:38:23.051 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:38:23.083 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:38:23.090 INFO:teuthology.orchestra.run.smithi145.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 10 MB/s | 4.1 MB 00:00 2024-03-11T19:38:23.113 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:38:23.141 INFO:teuthology.orchestra.run.smithi145.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-03-11T19:38:23.189 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:38:23.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:38:23.265 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:38:23.306 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:38:23.341 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:38:23.349 INFO:teuthology.orchestra.run.smithi145.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-03-11T19:38:23.419 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-11T19:38:23.420 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-11T19:38:23.430 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:38:23.520 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:38:23.563 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:38:23.595 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:38:23.625 INFO:teuthology.orchestra.run.smithi145.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-03-11T19:38:23.642 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:38:23.650 INFO:teuthology.orchestra.run.smithi145.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-03-11T19:38:23.683 INFO:teuthology.orchestra.run.smithi145.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 925 kB/s | 548 kB 00:00 2024-03-11T19:38:23.738 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:38:23.788 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:38:23.917 INFO:teuthology.orchestra.run.smithi145.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-03-11T19:38:24.034 INFO:teuthology.orchestra.run.smithi145.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-03-11T19:38:24.276 INFO:teuthology.orchestra.run.smithi145.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 188 kB/s | 67 kB 00:00 2024-03-11T19:38:24.468 INFO:teuthology.orchestra.run.smithi145.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-03-11T19:38:24.514 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:38:24.569 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:38:24.603 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:38:24.651 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:38:24.746 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:38:24.787 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi145.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-03-11T19:38:24.935 INFO:teuthology.orchestra.run.smithi145.stdout:(115/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-03-11T19:38:24.967 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:38:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:38:25.060 INFO:teuthology.orchestra.run.smithi145.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 925 kB/s | 548 kB 00:00 2024-03-11T19:38:25.070 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:38:25.202 INFO:teuthology.orchestra.run.smithi145.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 121 kB/s | 32 kB 00:00 2024-03-11T19:38:25.227 INFO:teuthology.orchestra.run.smithi145.stdout:(118/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2024-03-11T19:38:25.361 INFO:teuthology.orchestra.run.smithi145.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 50 kB/s | 15 kB 00:00 2024-03-11T19:38:25.628 INFO:teuthology.orchestra.run.smithi145.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-03-11T19:38:25.711 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:38:25.791 INFO:teuthology.orchestra.run.smithi145.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 641 kB/s | 358 kB 00:00 2024-03-11T19:38:25.831 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:38:25.870 INFO:teuthology.orchestra.run.smithi145.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-03-11T19:38:25.896 INFO:teuthology.orchestra.run.smithi145.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-11T19:38:26.296 INFO:teuthology.orchestra.run.smithi145.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 398 kB/s | 203 kB 00:00 2024-03-11T19:38:26.321 INFO:teuthology.orchestra.run.smithi145.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-03-11T19:38:26.491 INFO:teuthology.orchestra.run.smithi145.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 55 kB/s | 11 kB 00:00 2024-03-11T19:38:26.599 INFO:teuthology.orchestra.run.smithi145.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 63 kB/s | 18 kB 00:00 2024-03-11T19:38:26.783 INFO:teuthology.orchestra.run.smithi145.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 79 kB/s | 23 kB 00:00 2024-03-11T19:38:26.834 INFO:teuthology.orchestra.run.smithi145.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-03-11T19:38:26.870 INFO:teuthology.orchestra.run.smithi145.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-03-11T19:38:27.045 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:38:27.051 INFO:teuthology.orchestra.run.smithi145.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-03-11T19:38:27.127 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:38:27.157 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:38:27.189 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:38:27.209 INFO:teuthology.orchestra.run.smithi145.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-03-11T19:38:27.216 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:38:27.393 INFO:teuthology.orchestra.run.smithi145.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-03-11T19:38:27.518 INFO:teuthology.orchestra.run.smithi145.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-03-11T19:38:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:38:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:38:27.560 INFO:teuthology.orchestra.run.smithi145.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-03-11T19:38:27.628 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:38:27.660 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:38:27.696 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:38:27.785 INFO:teuthology.orchestra.run.smithi145.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-03-11T19:38:27.894 INFO:teuthology.orchestra.run.smithi145.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-03-11T19:38:27.919 INFO:teuthology.orchestra.run.smithi145.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 251 kB/s | 90 kB 00:00 2024-03-11T19:38:28.053 INFO:teuthology.orchestra.run.smithi145.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-03-11T19:38:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:38:28.153 INFO:teuthology.orchestra.run.smithi145.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-03-11T19:38:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:38:28.237 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:38:28.259 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:38:28.345 INFO:teuthology.orchestra.run.smithi145.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 443 kB/s | 188 kB 00:00 2024-03-11T19:38:28.395 INFO:teuthology.orchestra.run.smithi145.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-11T19:38:28.420 INFO:teuthology.orchestra.run.smithi145.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-03-11T19:38:28.687 INFO:teuthology.orchestra.run.smithi145.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-03-11T19:38:28.754 INFO:teuthology.orchestra.run.smithi145.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 269 kB/s | 90 kB 00:00 2024-03-11T19:38:28.905 INFO:teuthology.orchestra.run.smithi145.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-03-11T19:38:29.022 INFO:teuthology.orchestra.run.smithi145.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-03-11T19:38:29.272 INFO:teuthology.orchestra.run.smithi145.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi145.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-03-11T19:38:29.881 INFO:teuthology.orchestra.run.smithi145.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-03-11T19:38:29.923 INFO:teuthology.orchestra.run.smithi145.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-11T19:38:30.165 INFO:teuthology.orchestra.run.smithi145.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 9.5 MB/s | 2.3 MB 00:00 2024-03-11T19:38:30.365 INFO:teuthology.orchestra.run.smithi145.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 9.1 MB/s | 1.8 MB 00:00 2024-03-11T19:38:30.393 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:30.415 INFO:teuthology.orchestra.run.smithi145.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 852 kB/s | 42 kB 00:00 2024-03-11T19:38:30.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:30.590 INFO:teuthology.orchestra.run.smithi145.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.4 MB/s | 248 kB 00:00 2024-03-11T19:38:30.649 INFO:teuthology.orchestra.run.smithi145.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-11T19:38:30.691 INFO:teuthology.orchestra.run.smithi145.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 873 kB/s | 36 kB 00:00 2024-03-11T19:38:30.766 INFO:teuthology.orchestra.run.smithi145.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-11T19:38:30.909 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-11T19:38:30.968 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:31.001 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:38:31.009 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:38:31.041 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:38:31.073 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:38:31.073 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:38:31.074 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:38:31.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:31.102 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:38:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:31.244 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:38:31.351 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:38:31.405 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:38:31.444 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:38:31.546 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:38:31.667 INFO:teuthology.orchestra.run.smithi145.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 284 kB/s | 256 kB 00:00 2024-03-11T19:38:31.717 INFO:teuthology.orchestra.run.smithi145.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 462 kB/s | 23 kB 00:00 2024-03-11T19:38:31.808 INFO:teuthology.orchestra.run.smithi145.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-11T19:38:31.909 INFO:teuthology.orchestra.run.smithi145.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-11T19:38:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:38:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:32.017 INFO:teuthology.orchestra.run.smithi145.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-11T19:38:32.085 INFO:teuthology.orchestra.run.smithi145.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 958 kB/s | 64 kB 00:00 2024-03-11T19:38:32.138 INFO:teuthology.orchestra.run.smithi145.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 576 kB/s | 29 kB 00:00 2024-03-11T19:38:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:38:32.435 INFO:teuthology.orchestra.run.smithi145.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 4.0 MB/s | 1.2 MB 00:00 2024-03-11T19:38:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:38:32.727 INFO:teuthology.orchestra.run.smithi145.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.4 MB/s | 2.2 MB 00:00 2024-03-11T19:38:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:38:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:38:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:32.961 INFO:teuthology.orchestra.run.smithi145.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.8 MB/s | 2.3 MB 00:00 2024-03-11T19:38:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:33.060 INFO:teuthology.orchestra.run.smithi145.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.6 MB/s | 465 kB 00:00 2024-03-11T19:38:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:33.111 INFO:teuthology.orchestra.run.smithi145.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-03-11T19:38:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:38:33.244 INFO:teuthology.orchestra.run.smithi145.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2024-03-11T19:38:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:33.625 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:33.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:33.751 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:38:34.098 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:38:34.211 INFO:teuthology.orchestra.run.smithi145.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 77 kB/s | 74 kB 00:00 2024-03-11T19:38:34.258 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:34.340 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:34.584 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:35.029 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:35.218 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:38:35.336 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:38:35.381 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:38:35.420 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:38:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:38:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:38:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:38:35.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:38:35.923 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:38:35.989 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:38:36.037 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:38:36.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:38:36.195 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:38:36.229 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:38:36.304 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:38:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:38:36.454 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:38:36.514 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:38:36.613 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:38:36.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:38:36.867 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:38:37.517 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:38:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:38:37.753 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:38:37.839 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:37.865 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:37.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:38:37.993 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:38:38.048 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:38:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:38:38.504 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:38:38.745 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:38:38.888 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:38:38.957 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:38:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:38:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:38:39.316 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:38:39.410 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:38:39.447 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:38:39.486 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:38:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:38:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:38:39.680 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:38:39.742 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:38:39.938 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:38:40.028 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:38:40.067 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:38:40.127 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:38:40.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:38:40.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:38:40.394 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:38:40.431 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:38:40.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:38:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:38:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:38:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:38:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:38:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:38:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:38:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:38:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:38:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:38:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:38:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:38:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:38:41.872 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:38:41.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:38:41.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:38:42.040 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:38:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:38:42.207 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:38:42.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:38:42.295 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:38:42.357 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:38:42.399 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:38:42.520 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-03-11T19:38:42.940 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-03-11T19:38:42.941 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-03-11T19:38:42.941 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-03-11T19:38:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:43.063 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:38:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:38:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:38:43.115 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:43.115 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:38:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:38:43.174 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:38:43.235 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:38:43.384 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:38:43.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:38:43.810 INFO:teuthology.orchestra.run.smithi145.stdout:(173/174): librbd1-19.0.0-1607.g94a53970.el9.x8 221 kB/s | 3.1 MB 00:14 2024-03-11T19:38:44.417 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:38:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:38:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:38:44.510 INFO:teuthology.orchestra.run.smithi145.stdout:(174/174): librados2-19.0.0-1607.g94a53970.el9. 226 kB/s | 3.4 MB 00:15 2024-03-11T19:38:44.515 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:38:44.515 INFO:teuthology.orchestra.run.smithi145.stdout:Total 7.3 MB/s | 223 MB 00:30 2024-03-11T19:38:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:38:44.594 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:38:44.619 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:38:44.620 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:44.620 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:38:44.620 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:38:44.620 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:38:44.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:44.696 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:38:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:38:44.820 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:38:44.842 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:38:44.843 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:44.843 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:38:44.843 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:38:44.843 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:38:44.843 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:44.970 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-03-11T19:38:45.010 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:38:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:38:45.259 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-03-11T19:38:45.259 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-03-11T19:38:45.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:38:45.301 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:45.389 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:38:45.740 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:38:45.771 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:38:45.802 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:38:45.862 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:38:45.894 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:38:45.917 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:38:46.047 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:38:46.125 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:38:46.153 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:38:46.971 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-03-11T19:38:46.972 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-03-11T19:38:47.116 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:38:47.145 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:38:47.244 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:38:47.399 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:38:47.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:38:47.453 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:38:47.453 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:47.453 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:38:47.454 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:38:47.454 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:38:47.454 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:47.498 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:38:47.509 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:38:47.554 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:38:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:38:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:38:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:38:48.456 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:48.456 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:38:48.456 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:38:48.456 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:38:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:38:48.522 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:38:48.522 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:48.522 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:38:48.523 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:48.568 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:38:48.590 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:38:48.590 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:48.590 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:38:48.591 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:38:48.591 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:38:48.591 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:48.902 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:38:48.925 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:38:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:38:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:38:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:38:52.647 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:38:52.647 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:38:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:38:53.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:38:53.374 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:38:53.405 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:38:53.432 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:38:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:38:53.747 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:38:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-03-11T19:38:53.780 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:38:53.819 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:38:53.880 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:38:53.897 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:38:53.923 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:38:53.931 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:38:53.947 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:38:53.962 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:53.971 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:38:53.980 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:38:54.010 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:38:54.011 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:38:54.064 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:38:54.065 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:38:54.099 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:38:54.125 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:38:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:38:54.166 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:38:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:38:54.549 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:38:54.584 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:54.705 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:38:54.735 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:54.752 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:38:54.790 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:38:54.810 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:38:54.825 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:38:54.856 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:38:54.906 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:38:55.081 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:38:55.137 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:38:55.165 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:55.251 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:38:55.275 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:55.322 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:38:55.353 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:38:55.376 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:55.392 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:38:55.585 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:38:55.615 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:38:55.647 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:38:55.701 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:38:55.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:38:55.753 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:38:55.822 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:38:55.850 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:55.855 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:38:55.880 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:38:55.893 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:38:55.905 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:38:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:38:55.932 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:38:55.971 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:56.020 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:38:56.038 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:38:56.050 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:38:56.134 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:38:56.160 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:38:56.193 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:38:56.219 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:38:56.244 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:38:56.258 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:38:56.268 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:38:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:38:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:38:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:38:56.368 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:38:56.391 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:38:56.408 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:38:56.425 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:38:56.431 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:38:56.431 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:38:56.462 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:38:56.501 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:38:56.528 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:38:56.563 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:38:56.611 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:38:56.611 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:38:56.637 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:38:56.692 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:38:56.694 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:38:56.718 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:38:56.729 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:38:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:38:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:38:56.794 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:38:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:38:56.837 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:38:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:38:56.881 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:38:56.913 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:38:56.942 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:38:57.011 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:38:57.046 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:38:57.089 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:38:57.118 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:38:57.186 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:38:57.241 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:38:57.314 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:38:57.448 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:38:57.502 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:38:58.021 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:38:58.184 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:38:58.225 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:58.233 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:38:58.265 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:38:58.314 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:38:58.352 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:38:58.381 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:38:58.609 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:38:58.769 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:38:58.834 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:38:58.877 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:38:59.012 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:38:59.056 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:38:59.110 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:38:59.184 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:38:59.215 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:38:59.251 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:38:59.298 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:38:59.334 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:38:59.377 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:38:59.419 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:38:59.551 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:38:59.638 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:38:59.680 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:38:59.773 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:38:59.805 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:38:59.845 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:38:59.870 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:38:59.895 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:38:59.967 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:39:00.095 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:39:00.203 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:39:00.245 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:39:00.293 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:39:00.348 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:39:00.540 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:39:00.582 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:39:00.711 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:39:00.764 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:39:00.795 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:39:00.826 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:39:00.846 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:39:00.880 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:39:00.919 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:39:00.951 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:39:00.981 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:39:01.057 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:39:01.091 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:39:01.133 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:39:01.209 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:39:01.296 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:39:01.378 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:39:01.423 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:39:01.454 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:39:01.501 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:39:01.598 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:39:01.656 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:39:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:39:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:39:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:39:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:39:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:39:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:39:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:39:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:39:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:39:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:39:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:39:02.370 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:39:02.413 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:39:02.445 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:39:02.492 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:39:02.572 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:39:02.621 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:39:02.796 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:39:02.827 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:39:02.904 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:39:03.534 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:39:03.640 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:39:04.467 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:39:04.467 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:39:04.467 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-03-11T19:39:04.467 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.468 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:39:04.469 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.470 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:04.471 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.472 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:39:04.473 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:39:04.474 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:39:04.475 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:39:04.476 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:39:04.477 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:39:04.478 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:39:04.479 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:39:04.480 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:39:04.481 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:39:04.481 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-11T19:39:04.481 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-03-11T19:39:04.851 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:39:04.924 DEBUG:teuthology.parallel:result is None 2024-03-11T19:39:04.943 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:39:04.975 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:39:05.006 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:39:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:39:05.340 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:39:05.363 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:39:05.453 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:39:05.485 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:39:05.529 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-03-11T19:39:05.701 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:05.956 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:39:05.984 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:39:05.999 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:39:05.999 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:05.999 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:39:06.000 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:06.016 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:39:06.070 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:39:06.093 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:39:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:39:07.710 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:07.942 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:39:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:39:08.025 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:08.026 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:39:08.026 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:39:08.026 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:39:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:08.222 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:39:08.272 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:39:08.509 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:39:08.602 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:39:08.602 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:08.602 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:39:08.602 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:39:08.602 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:39:08.603 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:39:08.813 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:39:08.859 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:39:08.940 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:39:08.940 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:39:08.941 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:39:08.941 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:08.968 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:39:09.391 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:39:09.421 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:39:09.469 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:39:09.571 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:39:09.617 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:39:09.948 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:39:10.001 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:39:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:39:11.294 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:39:11.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:39:11.495 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:39:11.569 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:39:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:39:11.761 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:39:12.741 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:39:12.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:39:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:39:12.849 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:12.849 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:39:12.849 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:39:12.849 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:39:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:12.930 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:39:13.024 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:39:13.024 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:13.024 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:39:13.024 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:13.108 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:39:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:39:13.629 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:39:13.630 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:13.630 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:39:13.630 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:39:13.630 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:39:13.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:18.415 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:39:18.494 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:39:18.530 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:39:18.569 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:39:18.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:39:18.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:39:19.040 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:39:19.108 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:39:19.196 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:39:19.253 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:39:19.296 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:39:19.348 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:39:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:39:19.497 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:39:19.659 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:39:19.699 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:39:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:20.623 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:39:20.623 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-03-11T19:39:20.624 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:20.780 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:39:20.795 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:39:20.795 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:20.796 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:39:20.796 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:22.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:39:22.195 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:39:22.250 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:39:22.256 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:39:22.276 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:22.406 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:39:22.447 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:39:22.484 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:39:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:39:22.506 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:39:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:39:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:39:22.709 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:39:22.752 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:39:22.752 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:39:22.899 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:39:22.942 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:39:22.943 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:39:22.965 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:22.997 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:39:23.031 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:39:23.061 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:39:23.063 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:39:23.132 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:39:23.204 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:39:23.262 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:39:23.295 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:39:23.337 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:39:23.379 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:39:23.412 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:39:23.412 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:39:23.443 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:39:23.450 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:23.451 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:23.474 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:39:23.535 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:39:23.566 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:39:23.771 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:23.771 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:39:23.796 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:39:23.825 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:39:23.902 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:39:23.934 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:39:23.991 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:39:24.049 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:39:24.084 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:39:24.213 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:39:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:39:24.451 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-11T19:39:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:24.792 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:39:24.825 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:39:24.925 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:39:25.078 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:39:25.102 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:39:25.125 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:25.169 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:39:25.180 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:39:25.225 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:39:26.067 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:39:26.105 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:39:26.128 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:26.181 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:39:26.203 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:39:26.203 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:26.203 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:39:26.203 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:26.248 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:26.270 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:26.270 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:26.270 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:39:26.270 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:39:26.271 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:39:26.271 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:26.582 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:39:26.604 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:39:26.604 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:39:26.604 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:39:26.604 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:39:26.604 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:39:26.605 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:29.427 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:39:29.427 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:39:29.427 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:39:29.428 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:39:29.428 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:39:29.428 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:39:29.428 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:39:29.428 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:39:29.429 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:39:29.430 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:39:29.430 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:39:29.430 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:39:29.430 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:39:29.430 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:39:29.431 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:39:29.432 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:39:29.433 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:39:29.434 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:39:29.435 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:39:29.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:39:29.437 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:39:29.438 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:39:29.438 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:39:29.438 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:39:29.438 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:39:29.438 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:39:29.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:39:29.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:39:29.441 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:39:29.442 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:39:29.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:39:29.444 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:39:29.445 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:39:29.446 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:39:29.447 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:39:29.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:39:29.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:39:29.450 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:39:29.451 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:39:29.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:39:29.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:39:29.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:39:29.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:39:29.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:39:29.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:39:31.011 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:39:31.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:39:31.083 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:39:31.110 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:39:31.291 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:39:31.409 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:39:31.457 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:39:31.497 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:39:31.549 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:39:31.592 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:39:31.627 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:39:31.668 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:39:31.697 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:39:31.743 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:39:31.803 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:39:31.830 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:39:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:39:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:39:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:39:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:39:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:39:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:39:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:39:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:39:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:39:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:39:32.018 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-11T19:39:32.337 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:32.376 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:32.396 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:39:32.637 DEBUG:teuthology.parallel:result is None 2024-03-11T19:39:33.205 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:39:33.280 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:39:33.319 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:39:33.400 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:39:33.450 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:39:33.475 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:39:33.501 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:39:33.527 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:39:33.527 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:39:33.615 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:39:33.653 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:39:33.653 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:39:33.729 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:39:33.755 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:39:33.788 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:39:33.814 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:39:33.839 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:39:33.872 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:39:33.911 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:39:33.943 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:39:33.969 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:39:33.994 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:39:34.019 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:39:34.044 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:39:34.051 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:34.051 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:34.231 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:39:34.231 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:39:34.297 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:39:34.321 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:39:34.347 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:39:34.380 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:39:34.406 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:39:34.470 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:39:39.300 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:39:39.300 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:39:39.300 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:39:39.300 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:39:39.300 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:39:39.301 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:39:39.302 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:39:39.318 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:39:39.319 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:39:39.320 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:39:39.321 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:39:39.322 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:39:39.323 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:39:39.324 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:39:39.325 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:39:39.326 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:39:39.327 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:39:39.328 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:39:39.329 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:39:39.330 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:39:42.295 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:39:42.295 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:42.295 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.296 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:39:42.297 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.298 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:39:42.299 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:39:42.300 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:39:42.301 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:39:42.302 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:39:42.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:39:42.304 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:39:42.305 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:39:42.306 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:39:42.307 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:39:42.308 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:39:42.309 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:39:42.309 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:39:42.309 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:39:42.309 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-03-11T19:39:42.765 DEBUG:teuthology.parallel:result is None 2024-03-11T19:39:42.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:42.912 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:39:42.986 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:39:42.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:39:42.987 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:39:42.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:43.245 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:39:43.270 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:39:43.271 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:39:43.271 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:39:43.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:43.409 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:39:43.432 INFO:teuthology.orchestra.run.smithi145.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:39:43.433 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:39:43.433 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:39:43.435 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-11T19:39:43.435 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:39:43.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:39:43.540 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:39:43.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:39:43.575 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:39:43.575 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:39:43.608 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-11T19:39:43.608 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:39:43.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:39:43.708 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:39:43.802 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:39:43.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:39:43.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:39:43.901 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:39:43.901 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:39:43.932 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:39:44.001 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-11T19:39:44.002 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:39:44.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:39:44.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:39:44.237 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:39:44.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:39:44.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:39:44.331 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:39:44.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:39:44.361 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:39:44.430 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-11T19:39:44.430 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:39:44.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:39:44.529 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:39:44.620 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:39:44.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:39:44.646 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:39:44.713 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:39:44.713 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:39:44.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:39:44.815 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-11T19:39:44.911 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-11T19:39:44.911 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:44.912 INFO:tasks.cephadm:Cluster fsid is 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:39:44.912 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-11T19:39:44.912 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.122', 'mon.c': '172.21.15.145'} 2024-03-11T19:39:44.912 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-03-11T19:39:44.912 INFO:tasks.cephadm:First mgr is x 2024-03-11T19:39:44.912 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-11T19:39:44.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-03-11T19:39:45.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-03-11T19:39:45.034 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname $(hostname -s) 2024-03-11T19:39:45.065 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-11T19:39:45.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:45.282 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77431/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:08:19.449750', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77429/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:06:59.696700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-11T19:39:45.499 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 94a539707733325322964c6a187c6a1b41be1daf from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:39:45.500 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm 2024-03-11T19:39:45.501 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:46.255 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:39 /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:46.256 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:46.908 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:39 /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:46.909 DEBUG:teuthology.orchestra.run.smithi145:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:47.509 INFO:teuthology.orchestra.run.smithi145.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:39 /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:47.510 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:47.571 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:47.590 DEBUG:teuthology.orchestra.run.smithi145:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:39:47.626 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf on all hosts... 2024-03-11T19:39:47.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:39:47.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:39:47.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:39:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:39:48.501 INFO:teuthology.orchestra.run.smithi145.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:39:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:40:19.110 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-03-11T19:40:19.111 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout: "repo_digests": [ 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:40:19.408 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-03-11T19:40:19.409 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T19:40:22.734 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:40:22.791 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-03-11T19:40:22.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-03-11T19:40:22.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph 2024-03-11T19:40:22.963 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-03-11T19:40:23.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-03-11T19:40:23.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /etc/ceph 2024-03-11T19:40:23.121 INFO:tasks.cephadm:Writing seed config... 2024-03-11T19:40:23.122 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-11T19:40:23.123 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-11T19:40:23.124 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-11T19:40:23.125 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-11T19:40:23.126 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-11T19:40:23.126 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:40:23.126 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-11T19:40:23.183 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1ccf666c-dfdf-11ee-95c8-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-11T19:40:23.183 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service 2024-03-11T19:40:23.231 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x.service 2024-03-11T19:40:23.277 INFO:tasks.cephadm:Bootstrapping... 2024-03-11T19:40:23.277 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf -v bootstrap --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:40:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:40:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf', '-v', 'bootstrap', '--fsid', '1ccf666c-dfdf-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-03-11T19:40:23.608 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:40:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-11T19:40:23.609 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-03-11T19:40:23.659 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:40:23.659 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-03-11T19:40:23.659 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-03-11T19:40:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:40:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:40:23.698 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:40:23.698 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:40:23.723 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-11T19:40:23.743 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-11T19:40:23.744 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:40:23.744 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-03-11T19:40:23.800 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:40:23.800 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-11T19:40:23.800 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-03-11T19:40:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-03-11T19:40:23.815 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:40:23.815 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:40:23.829 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:40:23.830 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:40:23.854 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-11T19:40:23.875 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-11T19:40:23.875 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:40:23.875 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-03-11T19:40:23.875 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:23.875 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 139912788152272 on /run/cephadm/1ccf666c-dfdf-11ee-95c8-87774f69a715.lock 2024-03-11T19:40:23.876 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 139912788152272 acquired on /run/cephadm/1ccf666c-dfdf-11ee-95c8-87774f69a715.lock 2024-03-11T19:40:23.877 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-03-11T19:40:23.878 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-03-11T19:40:23.878 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-11T19:40:23.892 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-03-11T19:40:23.892 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-03-11T19:40:23.906 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-11T19:40:23.907 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-11T19:40:23.907 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-03-11T19:40:23.915 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-11T19:40:23.916 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-11T19:40:23.916 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:40:23.916 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-11T19:40:23.916 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-11T19:40:23.916 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:40:23.918 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-11T19:40:23.918 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-11T19:40:23.918 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-11T19:40:23.921 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-11T19:40:23.924 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:0b3c7bee4e66dab4b5376aaf0baf8ba5fc1c3e66c27dc601157782d617bd9dc6 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-11T19:40:24.543 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:40:24.544 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-11T19:40:25.822 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:40:25.823 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:40:25.823 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-03-11T19:40:36.089 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-03-11T19:40:36.089 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-03-11T19:40:37.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA0Xu9lOzbWIBAAVa004hdJgBvxVmDPG240UQ== 2024-03-11T19:40:38.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA1Xu9lsvUAKhAA6kK2wpcFi0btCwkF0p+h+g== 2024-03-11T19:40:39.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA2Xu9lLjvKMhAAr8SftWvm8Vf/yvqqVGLLNw== 2024-03-11T19:40:39.561 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-03-11T19:40:40.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:40:40.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-11T19:40:40.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:40:40.690 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-03-11T19:40:42.842 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-11T19:40:43.521 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-11T19:40:43.816 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target. 2024-03-11T19:40:43.816 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target. 2024-03-11T19:40:44.221 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a 2024-03-11T19:40:44.221 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service: Unit ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service not loaded. 2024-03-11T19:40:44.512 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target.wants/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@.service. 2024-03-11T19:40:45.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:44 smithi107 podman[31321]: 2024-03-11 19:40:44.783023071 +0000 UTC m=+0.030698842 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:40:45.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 podman[31321]: 2024-03-11 19:40:45.166431969 +0000 UTC m=+0.414107804 container init ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-11T19:40:45.281 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:40:45.281 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-11T19:40:45.294 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:40:45.295 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:40:45.295 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-11T19:40:45.295 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:40:45.296 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-03-11T19:40:45.296 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-03-11T19:40:45.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 podman[31321]: 2024-03-11 19:40:45.177338322 +0000 UTC m=+0.425014124 container start ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-11T19:40:45.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 bash[31321]: ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 2024-03-11T19:40:45.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 systemd[1]: Started Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:40:45.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 ceph-mon[31339]: mkfs 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:45.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:45 smithi107 ceph-mon[31339]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:40:46.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:40:46.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:40:46.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: fsmap 2024-03-11T19:40:46.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: osdmap e1: 0 total, 0 up, 0 in 2024-03-11T19:40:46.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: mgrmap e1: no daemons active 2024-03-11T19:40:46.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:46 smithi107 ceph-mon[31339]: from='client.? 172.21.15.107:0/1322361360' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-11T19:40:46.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-03-11T19:40:46.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:46.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-11T19:40:46.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:46.737 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-03-11T19:40:46.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.748178s) 2024-03-11T19:40:46.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-11T19:40:46.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-11T19:40:46.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:46.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-03-11T19:40:46.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-11T19:40:46.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-11T19:40:46.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-11T19:40:46.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-03-11T19:40:46.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.906+0000 7fbbf6e00700 1 Processor -- start 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.906+0000 7fbbf6e00700 1 -- start start 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.907+0000 7fbbf6e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.907+0000 7fbbf6e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0108050 con 0x7fbbf0107660 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.908+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.908+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60250/0 (socket says 172.21.15.107:60250) 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.908+0000 7fbbefe00700 1 -- 172.21.15.107:0/1417677819 learned_addr learned my addr 172.21.15.107:0/1417677819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:46.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.909+0000 7fbbefe00700 1 -- 172.21.15.107:0/1417677819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf01087c0 con 0x7fbbf0107660 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.909+0000 7fbbefe00700 1 --2- 172.21.15.107:0/1417677819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00098a0 tx=0x7fbbe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e6fd8df68809c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.909+0000 7fbbeea00700 1 -- 172.21.15.107:0/1417677819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe0004160 con 0x7fbbf0107660 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbeea00700 1 -- 172.21.15.107:0/1417677819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe00042c0 con 0x7fbbf0107660 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbeea00700 1 -- 172.21.15.107:0/1417677819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe00045a0 con 0x7fbbf0107660 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1417677819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 msgr2=0x7fbbf0107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 --2- 172.21.15.107:0/1417677819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00098a0 tx=0x7fbbe0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1417677819 shutdown_connections 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 --2- 172.21.15.107:0/1417677819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf0107a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:46.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1417677819 >> 172.21.15.107:0/1417677819 conn(0x7fbbf0102ce0 msgr2=0x7fbbf0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1417677819 shutdown_connections 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.910+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1417677819 wait complete. 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.911+0000 7fbbf6e00700 1 Processor -- start 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbf6e00700 1 -- start start 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbf6e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbf6e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019b050 con 0x7fbbf0107660 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbefe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60262/0 (socket says 172.21.15.107:60262) 2024-03-11T19:40:46.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.912+0000 7fbbefe00700 1 -- 172.21.15.107:0/1322361360 learned_addr learned my addr 172.21.15.107:0/1322361360 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbefe00700 1 -- 172.21.15.107:0/1322361360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009550 con 0x7fbbf0107660 2024-03-11T19:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbefe00700 1 --2- 172.21.15.107:0/1322361360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0004000 tx=0x7fbbe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe0011630 con 0x7fbbf0107660 2024-03-11T19:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0011790 con 0x7fbbf0107660 2024-03-11T19:40:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe0011a30 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf019b250 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.913+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf019b6f0 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.914+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbbe0021070 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.914+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbbe001b7d0 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.915+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf01947e0 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.918+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbbe001b990 con 0x7fbbf0107660 2024-03-11T19:40:46.750 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.997+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbbf0108da0 con 0x7fbbf0107660 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.998+0000 7fbbecc00700 1 -- 172.21.15.107:0/1322361360 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fbbe001bb30 con 0x7fbbf0107660 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 msgr2=0x7fbbf019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 --2- 172.21.15.107:0/1322361360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0004000 tx=0x7fbbe0004750 comp rx=0 tx=0).stop 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 shutdown_connections 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 --2- 172.21.15.107:0/1322361360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbf0107660 0x7fbbf019ab10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 >> 172.21.15.107:0/1322361360 conn(0x7fbbf0102ce0 msgr2=0x7fbbf0191920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:45.999+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 shutdown_connections 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:46.000+0000 7fbbf6e00700 1 -- 172.21.15.107:0/1322361360 wait complete. 2024-03-11T19:40:46.751 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-03-11T19:40:46.752 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-03-11T19:40:47.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:47 smithi107 ceph-mon[31339]: from='client.? 172.21.15.107:0/2731789536' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-11T19:40:47.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:47 smithi107 ceph-mon[31339]: from='client.? 172.21.15.107:0/2731789536' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:40:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.353+0000 7f5bba200700 1 Processor -- start 2024-03-11T19:40:48.103 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.353+0000 7f5bba200700 1 -- start start 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.353+0000 7f5bba200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.353+0000 7f5bba200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb410ead0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.354+0000 7f5bb3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.354+0000 7f5bb3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60268/0 (socket says 172.21.15.107:60268) 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.354+0000 7f5bb3400700 1 -- 172.21.15.107:0/4252826281 learned_addr learned my addr 172.21.15.107:0/4252826281 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.354+0000 7f5bb3400700 1 -- 172.21.15.107:0/4252826281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb410ee00 con 0x7f5bb410e0e0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.354+0000 7f5bb3400700 1 --2- 172.21.15.107:0/4252826281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098a0 tx=0x7f5ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865bd47bd470717 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.355+0000 7f5bb2000700 1 -- 172.21.15.107:0/4252826281 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba4004160 con 0x7f5bb410e0e0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.355+0000 7f5bb2000700 1 -- 172.21.15.107:0/4252826281 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba40042c0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.355+0000 7f5bb2000700 1 -- 172.21.15.107:0/4252826281 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba40045a0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.355+0000 7f5bba200700 1 -- 172.21.15.107:0/4252826281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 msgr2=0x7f5bb410e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.355+0000 7f5bba200700 1 --2- 172.21.15.107:0/4252826281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40098a0 tx=0x7f5ba4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.356+0000 7f5bba200700 1 -- 172.21.15.107:0/4252826281 shutdown_connections 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.356+0000 7f5bba200700 1 --2- 172.21.15.107:0/4252826281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb410e500 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.356+0000 7f5bba200700 1 -- 172.21.15.107:0/4252826281 >> 172.21.15.107:0/4252826281 conn(0x7f5bb407cdf0 msgr2=0x7f5bb407f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.356+0000 7f5bba200700 1 -- 172.21.15.107:0/4252826281 shutdown_connections 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.356+0000 7f5bba200700 1 -- 172.21.15.107:0/4252826281 wait complete. 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bba200700 1 Processor -- start 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bba200700 1 -- start start 2024-03-11T19:40:48.105 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bba200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bba200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41a1640 con 0x7f5bb410e0e0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bb3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bb3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60278/0 (socket says 172.21.15.107:60278) 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.357+0000 7f5bb3400700 1 -- 172.21.15.107:0/2731789536 learned_addr learned my addr 172.21.15.107:0/2731789536 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5bb3400700 1 -- 172.21.15.107:0/2731789536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4009550 con 0x7f5bb410e0e0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5bb3400700 1 --2- 172.21.15.107:0/2731789536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4004000 tx=0x7f5ba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba4011630 con 0x7f5bb410e0e0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4011790 con 0x7f5bb410e0e0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba4011a30 con 0x7f5bb410e0e0 2024-03-11T19:40:48.106 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.358+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb41a1840 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.359+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb41a1ce0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.359+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5ba4021070 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.359+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5ba401b7d0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.360+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb419add0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.363+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5ba401b990 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.438+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5bb410f110 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.445+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f5ba401bbe0 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.446+0000 7f5babe00700 1 -- 172.21.15.107:0/2731789536 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ba401bd80 con 0x7f5bb410e0e0 2024-03-11T19:40:48.107 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.447+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 msgr2=0x7f5bb41a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.447+0000 7f5bba200700 1 --2- 172.21.15.107:0/2731789536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4004000 tx=0x7f5ba4004750 comp rx=0 tx=0).stop 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.448+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 shutdown_connections 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.448+0000 7f5bba200700 1 --2- 172.21.15.107:0/2731789536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410e0e0 0x7f5bb41a1100 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.448+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 >> 172.21.15.107:0/2731789536 conn(0x7f5bb407cdf0 msgr2=0x7f5bb4197f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.448+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 shutdown_connections 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:47.448+0000 7f5bba200700 1 -- 172.21.15.107:0/2731789536 wait complete. 2024-03-11T19:40:48.108 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-03-11T19:40:49.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:48 smithi107 ceph-mon[31339]: from='client.? 172.21.15.107:0/1018196349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.675+0000 7fa366200700 1 Processor -- start 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.675+0000 7fa366200700 1 -- start start 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.675+0000 7fa366200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.675+0000 7fa366200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360107fd0 con 0x7fa3601075e0 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.676+0000 7fa35f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.676+0000 7fa35f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60294/0 (socket says 172.21.15.107:60294) 2024-03-11T19:40:49.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.676+0000 7fa35f400700 1 -- 172.21.15.107:0/2592924100 learned_addr learned my addr 172.21.15.107:0/2592924100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.676+0000 7fa35f400700 1 -- 172.21.15.107:0/2592924100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360108790 con 0x7fa3601075e0 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.677+0000 7fa35f400700 1 --2- 172.21.15.107:0/2592924100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3500098a0 tx=0x7fa350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65d8e6a56ef16a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.677+0000 7fa35e000700 1 -- 172.21.15.107:0/2592924100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa350004160 con 0x7fa3601075e0 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.677+0000 7fa35e000700 1 -- 172.21.15.107:0/2592924100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3500042c0 con 0x7fa3601075e0 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.677+0000 7fa35e000700 1 -- 172.21.15.107:0/2592924100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa350011ac0 con 0x7fa3601075e0 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 -- 172.21.15.107:0/2592924100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 msgr2=0x7fa360107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 --2- 172.21.15.107:0/2592924100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3500098a0 tx=0x7fa350009bb0 comp rx=0 tx=0).stop 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 -- 172.21.15.107:0/2592924100 shutdown_connections 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 --2- 172.21.15.107:0/2592924100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa360107a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 -- 172.21.15.107:0/2592924100 >> 172.21.15.107:0/2592924100 conn(0x7fa360102c40 msgr2=0x7fa360105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 -- 172.21.15.107:0/2592924100 shutdown_connections 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.678+0000 7fa366200700 1 -- 172.21.15.107:0/2592924100 wait complete. 2024-03-11T19:40:49.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.679+0000 7fa366200700 1 Processor -- start 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.679+0000 7fa366200700 1 -- start start 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.679+0000 7fa366200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.679+0000 7fa366200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36007d0d0 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.680+0000 7fa35f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.680+0000 7fa35f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60310/0 (socket says 172.21.15.107:60310) 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.680+0000 7fa35f400700 1 -- 172.21.15.107:0/1018196349 learned_addr learned my addr 172.21.15.107:0/1018196349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.680+0000 7fa35f400700 1 -- 172.21.15.107:0/1018196349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009550 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.680+0000 7fa35f400700 1 --2- 172.21.15.107:0/1018196349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa350000c00 tx=0x7fa350011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa350003aa0 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa350003c00 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36007b2c0 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3500248a0 con 0x7fa3601075e0 2024-03-11T19:40:49.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36007b760 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa35002d3f0 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.681+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa350019780 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.682+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36010b9c0 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.685+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa35001ac60 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.761+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa36010bbd0 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.762+0000 7fa357e00700 1 -- 172.21.15.107:0/1018196349 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa350035090 con 0x7fa3601075e0 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 msgr2=0x7fa36007cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 --2- 172.21.15.107:0/1018196349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa350000c00 tx=0x7fa350011b40 comp rx=0 tx=0).stop 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 shutdown_connections 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 --2- 172.21.15.107:0/1018196349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3601075e0 0x7fa36007cb90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 >> 172.21.15.107:0/1018196349 conn(0x7fa360102c40 msgr2=0x7fa3601037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.763+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 shutdown_connections 2024-03-11T19:40:49.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:48.764+0000 7fa366200700 1 -- 172.21.15.107:0/1018196349 wait complete. 2024-03-11T19:40:49.403 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-03-11T19:40:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:49 smithi107 systemd[1]: Stopping Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:40:50.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a[31335]: 2024-03-11T19:40:49.867+0000 7f0e25000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:40:50.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a[31335]: 2024-03-11T19:40:49.867+0000 7f0e25000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-11T19:40:50.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:50 smithi107 podman[31582]: 2024-03-11 19:40:50.062939299 +0000 UTC m=+0.330928541 container died ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:40:50.765 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:50 smithi107 podman[31582]: 2024-03-11 19:40:50.452013769 +0000 UTC m=+0.720003038 container cleanup ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-94a5397) 2024-03-11T19:40:50.765 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:50 smithi107 bash[31582]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a 2024-03-11T19:40:51.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:50 smithi107 podman[31595]: 2024-03-11 19:40:50.761062872 +0000 UTC m=+0.681631336 container remove ccde271789725647b4f2ed4364778d33ad361445f9d75e1fa075f0685e292c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-11T19:40:51.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-11T19:40:51.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 systemd[1]: Stopped Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:40:51.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service: Consumed 1.785s CPU time. 2024-03-11T19:40:51.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 systemd[1]: Starting Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:40:51.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 podman[31692]: 2024-03-11 19:40:51.366965276 +0000 UTC m=+0.038934294 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:40:51.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:51 smithi107 podman[31692]: 2024-03-11 19:40:51.478532572 +0000 UTC m=+0.150501617 container create d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:40:52.228 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 podman[31692]: 2024-03-11 19:40:52.13217429 +0000 UTC m=+0.804143385 container init d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 podman[31692]: 2024-03-11 19:40:52.142523207 +0000 UTC m=+0.814492227 container start d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: set uid:gid to 167:167 (ceph:ceph) 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: pidfile_write: ignore empty --pid-file 2024-03-11T19:40:52.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: load: jerasure load: lrc 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: RocksDB version: 7.9.2 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Git sha 0 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: DB SUMMARY 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: DB Session ID: VI0TGH1TO7US7OC9QNE0 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: CURRENT file: CURRENT 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: IDENTITY file: IDENTITY 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100338 ; 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.error_if_exists: 0 2024-03-11T19:40:52.380 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.create_if_missing: 0 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.paranoid_checks: 1 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.env: 0x55e27c787ae0 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.fs: PosixFileSystem 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.info_log: 0x55e27e4cf900 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_file_opening_threads: 16 2024-03-11T19:40:52.381 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.statistics: (nil) 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.use_fsync: 0 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_log_file_size: 0 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.keep_log_file_num: 1000 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.recycle_log_file_num: 0 2024-03-11T19:40:52.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_fallocate: 1 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_mmap_reads: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_mmap_writes: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.use_direct_reads: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.create_missing_column_families: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.db_log_dir: 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.wal_dir: 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-11T19:40:52.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.advise_random_on_open: 1 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.db_write_buffer_size: 0 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.write_buffer_manager: 0x55e27e4a3400 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.rate_limiter: (nil) 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-11T19:40:52.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.wal_recovery_mode: 2 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enable_thread_tracking: 0 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enable_pipelined_write: 0 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.unordered_write: 0 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.row_cache: None 2024-03-11T19:40:52.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.wal_filter: None 2024-03-11T19:40:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-11T19:40:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_ingest_behind: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.two_write_queues: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.manual_wal_flush: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.wal_compression: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.atomic_flush: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-11T19:40:52.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.log_readahead_size: 0 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.best_efforts_recovery: 0 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.allow_data_in_errors: 0 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.db_host_id: __hostname__ 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_background_jobs: 2 2024-03-11T19:40:52.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_background_compactions: -1 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_subcompactions: 1 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_total_wal_size: 0 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-11T19:40:52.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_open_files: -1 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bytes_per_sync: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_readahead_size: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_background_flushes: -1 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Compression algorithms supported: 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kZSTD supported: 0 2024-03-11T19:40:52.390 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kXpressCompression supported: 0 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kLZ4HCCompression supported: 1 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kZlibCompression supported: 1 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kSnappyCompression supported: 1 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kLZ4Compression supported: 1 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: kBZip2Compression supported: 0 2024-03-11T19:40:52.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.merge_operator: 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_filter: None 2024-03-11T19:40:52.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_filter_factory: None 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.sst_partitioner_factory: None 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e27e4cfa00) 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-03-11T19:40:52.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x55e27e4d3090 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-03-11T19:40:52.395 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-03-11T19:40:52.396 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-03-11T19:40:52.397 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.write_buffer_size: 33554432 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_write_buffer_number: 2 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression: NoCompression 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression: Disabled 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.prefix_extractor: nullptr 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-11T19:40:52.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.num_levels: 7 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:40:52.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.level: 32767 2024-03-11T19:40:52.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.strategy: 0 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.enabled: false 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-11T19:40:52.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.target_file_size_base: 67108864 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-11T19:40:52.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-11T19:40:52.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-11T19:40:52.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-11T19:40:52.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.arena_block_size: 1048576 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.disable_auto_compactions: 0 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-11T19:40:52.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.table_properties_collectors: 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.inplace_update_support: 0 2024-03-11T19:40:52.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.bloom_locality: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.max_successive_merges: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.paranoid_file_checks: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.force_consistency_checks: 1 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.report_bg_io_stats: 0 2024-03-11T19:40:52.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.ttl: 2592000 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enable_blob_files: false 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.min_blob_size: 0 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_file_size: 268435456 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-11T19:40:52.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.blob_file_starting_level: 0 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d2393b86-b04e-4ac6-8e03-2dd5d9c3b1e7 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710186052199247, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710186052201881, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95467, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93453, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13912, "raw_average_key_size": 50, "raw_value_size": 85897, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710186052, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d2393b86-b04e-4ac6-8e03-2dd5d9c3b1e7", "db_session_id": "VI0TGH1TO7US7OC9QNE0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-11T19:40:52.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710186052202085, "job": 1, "event": "recovery_finished"} 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e27e5c4000 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: DB pointer 0x55e27e5ae000 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-11T19:40:52.409 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-11T19:40:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 95.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 62.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 95.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 62.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 62.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 62.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-11T19:40:52.411 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 9.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 9.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-11T19:40:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x55e27e4d3090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???) e1 preinit fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).mds e1 new map 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).mds e1 print_map 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-03-11T19:40:52.413 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-11T19:40:52.414 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 bash[31692]: d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 systemd[1]: Started Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: fsmap 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: osdmap e1: 0 total, 0 up, 0 in 2024-03-11T19:40:52.415 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:52 smithi107 ceph-mon[31712]: mgrmap e1: no daemons active 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.843+0000 7fbe86a00700 1 Processor -- start 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.843+0000 7fbe86a00700 1 -- start start 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.844+0000 7fbe86a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.844+0000 7fbe86a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8007b910 con 0x7fbe8007ce90 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.844+0000 7fbe7fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:53.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.844+0000 7fbe7fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60318/0 (socket says 172.21.15.107:60318) 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.844+0000 7fbe7fe00700 1 -- 172.21.15.107:0/1587434816 learned_addr learned my addr 172.21.15.107:0/1587434816 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.845+0000 7fbe7fe00700 1 -- 172.21.15.107:0/1587434816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8007ba50 con 0x7fbe8007ce90 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.845+0000 7fbe7fe00700 1 --2- 172.21.15.107:0/1587434816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbe700098a0 tx=0x7fbe70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0aa1215386d4f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe7ea00700 1 -- 172.21.15.107:0/1587434816 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe70004160 con 0x7fbe8007ce90 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe7ea00700 1 -- 172.21.15.107:0/1587434816 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe700042c0 con 0x7fbe8007ce90 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe7ea00700 1 -- 172.21.15.107:0/1587434816 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe70011ac0 con 0x7fbe8007ce90 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe86a00700 1 -- 172.21.15.107:0/1587434816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 msgr2=0x7fbe8007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe86a00700 1 --2- 172.21.15.107:0/1587434816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbe700098a0 tx=0x7fbe70009bb0 comp rx=0 tx=0).stop 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe86a00700 1 -- 172.21.15.107:0/1587434816 shutdown_connections 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.846+0000 7fbe86a00700 1 --2- 172.21.15.107:0/1587434816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8007b340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.847+0000 7fbe86a00700 1 -- 172.21.15.107:0/1587434816 >> 172.21.15.107:0/1587434816 conn(0x7fbe80102be0 msgr2=0x7fbe80105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.847+0000 7fbe86a00700 1 -- 172.21.15.107:0/1587434816 shutdown_connections 2024-03-11T19:40:53.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.847+0000 7fbe86a00700 1 -- 172.21.15.107:0/1587434816 wait complete. 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe86a00700 1 Processor -- start 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe86a00700 1 -- start start 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe86a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe86a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8019da60 con 0x7fbe8007ce90 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe7fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe7fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60330/0 (socket says 172.21.15.107:60330) 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.848+0000 7fbe7fe00700 1 -- 172.21.15.107:0/353352962 learned_addr learned my addr 172.21.15.107:0/353352962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.849+0000 7fbe7fe00700 1 -- 172.21.15.107:0/353352962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe70009550 con 0x7fbe8007ce90 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.849+0000 7fbe7fe00700 1 --2- 172.21.15.107:0/353352962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009130 tx=0x7fbe70011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.849+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe70003aa0 con 0x7fbe8007ce90 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe70003c00 con 0x7fbe8007ce90 2024-03-11T19:40:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8019dc60 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe70024800 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8019e100 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbe7002d3f0 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.850+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe700197b0 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.851+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8010d970 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.854+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbe7001abd0 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.925+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fbe8010db80 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.928+0000 7fbe7cc00700 1 -- 172.21.15.107:0/353352962 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbe7001ad70 con 0x7fbe8007ce90 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.929+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 msgr2=0x7fbe8019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.929+0000 7fbe86a00700 1 --2- 172.21.15.107:0/353352962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009130 tx=0x7fbe70011b40 comp rx=0 tx=0).stop 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.929+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 shutdown_connections 2024-03-11T19:40:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.929+0000 7fbe86a00700 1 --2- 172.21.15.107:0/353352962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe8007ce90 0x7fbe8019d520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.929+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 >> 172.21.15.107:0/353352962 conn(0x7fbe80102be0 msgr2=0x7fbe80103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.930+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 shutdown_connections 2024-03-11T19:40:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:52.930+0000 7fbe86a00700 1 -- 172.21.15.107:0/353352962 wait complete. 2024-03-11T19:40:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-11T19:40:53.682 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:40:53.682 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-03-11T19:40:53.684 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-11T19:40:53.685 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-11T19:40:53.934 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:53 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/353352962' entity='client.admin' 2024-03-11T19:40:54.168 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x 2024-03-11T19:40:54.168 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x.service: Unit ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x.service not loaded. 2024-03-11T19:40:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715.target.wants/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@.service. 2024-03-11T19:40:54.793 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:54 smithi107 systemd[1]: Starting Ceph mgr.x for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:40:55.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:54 smithi107 podman[31937]: 2024-03-11 19:40:54.693272584 +0000 UTC m=+0.035017285 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:40:55.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:54 smithi107 podman[31937]: 2024-03-11 19:40:54.805796082 +0000 UTC m=+0.147540820 container create 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:40:55.175 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:40:55.175 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-11T19:40:55.191 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:40:55.191 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:40:55.191 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-11T19:40:55.191 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:40:55.213 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:40:55.213 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-11T19:40:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:40:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:40:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-11T19:40:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-11T19:40:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-03-11T19:40:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-03-11T19:40:55.385 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:55 smithi107 podman[31937]: 2024-03-11 19:40:55.059241586 +0000 UTC m=+0.400986331 container init 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T19:40:55.385 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:55 smithi107 podman[31937]: 2024-03-11 19:40:55.069031591 +0000 UTC m=+0.410776315 container start 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-11T19:40:55.385 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:55 smithi107 bash[31937]: 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 2024-03-11T19:40:55.385 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:55 smithi107 systemd[1]: Started Ceph mgr.x for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:40:55.974 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:55 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:55.860+0000 7f762bfad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:40:56.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:56 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/373365746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:40:56.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:56 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:55.999+0000 7f762bfad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:40:57.141 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:56 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:56.833+0000 7f762bfad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:40:57.141 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:56 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:56.955+0000 7f762bfad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:40:57.141 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.135+0000 7f762bfad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:40:57.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:40:57.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:40:57.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:40:57.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:40:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:40:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:40:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:40:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:40:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:40:57.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:40:57.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:40:45.252069+0000", 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.877+0000 7f9e06e00700 1 Processor -- start 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.877+0000 7f9e06e00700 1 -- start start 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.877+0000 7f9e06e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.877+0000 7f9e06e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0007ae90 con 0x7f9e00105e70 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.878+0000 7f9dffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.878+0000 7f9dffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52946/0 (socket says 172.21.15.107:52946) 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.878+0000 7f9dffe00700 1 -- 172.21.15.107:0/746566537 learned_addr learned my addr 172.21.15.107:0/746566537 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.878+0000 7f9dffe00700 1 -- 172.21.15.107:0/746566537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e001067d0 con 0x7f9e00105e70 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.879+0000 7f9dffe00700 1 --2- 172.21.15.107:0/746566537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9df00098a0 tx=0x7f9df0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8230038635019d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.879+0000 7f9dfea00700 1 -- 172.21.15.107:0/746566537 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df0004160 con 0x7f9e00105e70 2024-03-11T19:40:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.879+0000 7f9dfea00700 1 -- 172.21.15.107:0/746566537 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9df00042c0 con 0x7f9e00105e70 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.879+0000 7f9dfea00700 1 -- 172.21.15.107:0/746566537 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df0011ac0 con 0x7f9e00105e70 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 -- 172.21.15.107:0/746566537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 msgr2=0x7f9e00106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 --2- 172.21.15.107:0/746566537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9df00098a0 tx=0x7f9df0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 -- 172.21.15.107:0/746566537 shutdown_connections 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 --2- 172.21.15.107:0/746566537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e00106290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 -- 172.21.15.107:0/746566537 >> 172.21.15.107:0/746566537 conn(0x7f9e00101ad0 msgr2=0x7f9e00103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 -- 172.21.15.107:0/746566537 shutdown_connections 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.880+0000 7f9e06e00700 1 -- 172.21.15.107:0/746566537 wait complete. 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.881+0000 7f9e06e00700 1 Processor -- start 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9e06e00700 1 -- start start 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9e06e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9e06e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0019db40 con 0x7f9e00105e70 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9dffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9dffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52958/0 (socket says 172.21.15.107:52958) 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.882+0000 7f9dffe00700 1 -- 172.21.15.107:0/373365746 learned_addr learned my addr 172.21.15.107:0/373365746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9dffe00700 1 -- 172.21.15.107:0/373365746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0009550 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9dffe00700 1 --2- 172.21.15.107:0/373365746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011a70 tx=0x7f9df0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df00039c0 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9df0003b20 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df0024610 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.883+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0019dd40 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.884+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0019e1e0 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.884+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9df0024800 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.884+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9df0019700 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.885+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e00197290 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.887+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9df001a7c0 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.967+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9e00107930 con 0x7f9e00105e70 2024-03-11T19:40:57.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.968+0000 7f9dfcc00700 1 -- 172.21.15.107:0/373365746 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f9df0035090 con 0x7f9e00105e70 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 msgr2=0x7f9e0019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 --2- 172.21.15.107:0/373365746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011a70 tx=0x7f9df0011b50 comp rx=0 tx=0).stop 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 shutdown_connections 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 --2- 172.21.15.107:0/373365746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e00105e70 0x7f9e0019d600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 >> 172.21.15.107:0/373365746 conn(0x7f9e00101ad0 msgr2=0x7f9e001943d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 shutdown_connections 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:55.970+0000 7f9e06e00700 1 -- 172.21.15.107:0/373365746 wait complete. 2024-03-11T19:40:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-03-11T19:40:57.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.261+0000 7f762bfad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:40:57.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.369+0000 7f762bfad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:40:57.963 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.707+0000 7f762bfad200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:40:57.964 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.846+0000 7f762bfad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:40:58.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:57.958+0000 7f762bfad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:40:58.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:58.079+0000 7f762bfad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:40:58.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:58.307+0000 7f762bfad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:40:59.877 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:59 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:59.621+0000 7f762bfad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:40:59.877 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:40:59 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:40:59.801+0000 7f762bfad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:41:00.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:40:59 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2168606349' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:41:00.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:00.235+0000 7f762bfad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:41:00.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:00.353+0000 7f762bfad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:41:00.590 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:00.590 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:41:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:41:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-11T19:41:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:41:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:41:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:41:00.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:41:00.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:40:45.252069+0000", 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.782+0000 7f6c53200700 1 Processor -- start 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.782+0000 7f6c53200700 1 -- start start 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.782+0000 7f6c53200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:00.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.782+0000 7f6c53200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c07b840 con 0x7f6c4c07ce50 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.783+0000 7f6c4be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.783+0000 7f6c4be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52966/0 (socket says 172.21.15.107:52966) 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.783+0000 7f6c4be00700 1 -- 172.21.15.107:0/4235753353 learned_addr learned my addr 172.21.15.107:0/4235753353 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.783+0000 7f6c4be00700 1 -- 172.21.15.107:0/4235753353 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c07b980 con 0x7f6c4c07ce50 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.783+0000 7f6c4be00700 1 --2- 172.21.15.107:0/4235753353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04130052e6e289c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c4aa00700 1 -- 172.21.15.107:0/4235753353 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c004160 con 0x7f6c4c07ce50 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c4aa00700 1 -- 172.21.15.107:0/4235753353 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3c0042c0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c4aa00700 1 -- 172.21.15.107:0/4235753353 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c011ac0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c53200700 1 -- 172.21.15.107:0/4235753353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 msgr2=0x7f6c4c07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c53200700 1 --2- 172.21.15.107:0/4235753353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c53200700 1 -- 172.21.15.107:0/4235753353 shutdown_connections 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c53200700 1 --2- 172.21.15.107:0/4235753353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c07b300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.784+0000 7f6c53200700 1 -- 172.21.15.107:0/4235753353 >> 172.21.15.107:0/4235753353 conn(0x7f6c4c102be0 msgr2=0x7f6c4c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:00.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.785+0000 7f6c53200700 1 -- 172.21.15.107:0/4235753353 shutdown_connections 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.785+0000 7f6c53200700 1 -- 172.21.15.107:0/4235753353 wait complete. 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.786+0000 7f6c53200700 1 Processor -- start 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.786+0000 7f6c53200700 1 -- start start 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c53200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c53200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c19daa0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c4be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c4be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52980/0 (socket says 172.21.15.107:52980) 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c4be00700 1 -- 172.21.15.107:0/2168606349 learned_addr learned my addr 172.21.15.107:0/2168606349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.787+0000 7f6c4be00700 1 -- 172.21.15.107:0/2168606349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009550 con 0x7f6c4c07ce50 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c4be00700 1 --2- 172.21.15.107:0/2168606349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009130 tx=0x7f6c3c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c003aa0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3c003c00 con 0x7f6c4c07ce50 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c024800 con 0x7f6c4c07ce50 2024-03-11T19:41:00.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c19dca0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c19e140 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.788+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6c3c02d3f0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.789+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c3c019760 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.789+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c10d950 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.792+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6c3c024e00 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.871+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6c4c1a0cb0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.872+0000 7f6c48c00700 1 -- 172.21.15.107:0/2168606349 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f6c3c01abd0 con 0x7f6c4c07ce50 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 msgr2=0x7f6c4c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 --2- 172.21.15.107:0/2168606349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009130 tx=0x7f6c3c011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 shutdown_connections 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 --2- 172.21.15.107:0/2168606349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4c07ce50 0x7f6c4c19d560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 >> 172.21.15.107:0/2168606349 conn(0x7f6c4c102be0 msgr2=0x7f6c4c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 shutdown_connections 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:40:59.874+0000 7f6c53200700 1 -- 172.21.15.107:0/2168606349 wait complete. 2024-03-11T19:41:00.606 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-03-11T19:41:00.848 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:00.609+0000 7f762bfad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:01.211 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:00.842+0000 7f762bfad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:41:01.211 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:01 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:01.205+0000 7f762bfad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:01.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:01 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:01.526+0000 7f762bfad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:41:02.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:01 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:01.825+0000 7f762bfad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:41:03.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:02 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:02.845+0000 7f762bfad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:41:03.566 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/932075665' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:41:03.976 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:03 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:03.560+0000 7f762bfad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:41:03.976 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:03 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:03.689+0000 7f762bfad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:41:04.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:04.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:04.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:41:04.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.082 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:41:04.082 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:41:04.082 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:41:04.082 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:41:04.082 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:41:04.083 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:41:04.083 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:41:04.083 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:41:04.083 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:41:04.083 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.084 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:41:04.085 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:41:04.085 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:41:04.085 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:41:04.085 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:41:04.085 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:41:04.086 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:04.087 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:40:45.252069+0000", 2024-03-11T19:41:04.087 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:04.087 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:04.087 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:41:04.087 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.194+0000 7f1f23a00700 1 Processor -- start 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.194+0000 7f1f23a00700 1 -- start start 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.194+0000 7f1f23a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.194+0000 7f1f23a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c07aed0 con 0x7f1f1c108040 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.195+0000 7f1f21000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:04.088 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.195+0000 7f1f21000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52984/0 (socket says 172.21.15.107:52984) 2024-03-11T19:41:04.089 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.195+0000 7f1f21000700 1 -- 172.21.15.107:0/515699569 learned_addr learned my addr 172.21.15.107:0/515699569 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:04.089 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.195+0000 7f1f21000700 1 -- 172.21.15.107:0/515699569 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c10a970 con 0x7f1f1c108040 2024-03-11T19:41:04.089 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.195+0000 7f1f21000700 1 --2- 172.21.15.107:0/515699569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f100098a0 tx=0x7f1f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01e7153dccd2e11 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:04.089 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.196+0000 7f1f1b400700 1 -- 172.21.15.107:0/515699569 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f10004160 con 0x7f1f1c108040 2024-03-11T19:41:04.089 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.196+0000 7f1f1b400700 1 -- 172.21.15.107:0/515699569 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f100042c0 con 0x7f1f1c108040 2024-03-11T19:41:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.196+0000 7f1f1b400700 1 -- 172.21.15.107:0/515699569 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f10011ac0 con 0x7f1f1c108040 2024-03-11T19:41:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.196+0000 7f1f23a00700 1 -- 172.21.15.107:0/515699569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 msgr2=0x7f1f1c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.196+0000 7f1f23a00700 1 --2- 172.21.15.107:0/515699569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f100098a0 tx=0x7f1f10009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.197+0000 7f1f23a00700 1 -- 172.21.15.107:0/515699569 shutdown_connections 2024-03-11T19:41:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.197+0000 7f1f23a00700 1 --2- 172.21.15.107:0/515699569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c10a430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:04.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.197+0000 7f1f23a00700 1 -- 172.21.15.107:0/515699569 >> 172.21.15.107:0/515699569 conn(0x7f1f1c101ad0 msgr2=0x7f1f1c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:04.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.197+0000 7f1f23a00700 1 -- 172.21.15.107:0/515699569 shutdown_connections 2024-03-11T19:41:04.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.197+0000 7f1f23a00700 1 -- 172.21.15.107:0/515699569 wait complete. 2024-03-11T19:41:04.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.198+0000 7f1f23a00700 1 Processor -- start 2024-03-11T19:41:04.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f23a00700 1 -- start start 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f23a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f23a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c19b940 con 0x7f1f1c108040 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f21000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f21000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52992/0 (socket says 172.21.15.107:52992) 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.199+0000 7f1f21000700 1 -- 172.21.15.107:0/932075665 learned_addr learned my addr 172.21.15.107:0/932075665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:04.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f21000700 1 -- 172.21.15.107:0/932075665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10009550 con 0x7f1f1c108040 2024-03-11T19:41:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f21000700 1 --2- 172.21.15.107:0/932075665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f10011a70 tx=0x7f1f10011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f100039c0 con 0x7f1f1c108040 2024-03-11T19:41:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f10003b20 con 0x7f1f1c108040 2024-03-11T19:41:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c19bb40 con 0x7f1f1c108040 2024-03-11T19:41:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f10024610 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.200+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c19bfe0 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.201+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1f10024c00 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.201+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f10019a00 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.201+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c10b7d0 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.204+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1f1001aa10 con 0x7f1f1c108040 2024-03-11T19:41:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.278+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1f1c10b9b0 con 0x7f1f1c108040 2024-03-11T19:41:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.279+0000 7f1f19600700 1 -- 172.21.15.107:0/932075665 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f1f1001abb0 con 0x7f1f1c108040 2024-03-11T19:41:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.280+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 msgr2=0x7f1f1c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.280+0000 7f1f23a00700 1 --2- 172.21.15.107:0/932075665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f10011a70 tx=0x7f1f10011b50 comp rx=0 tx=0).stop 2024-03-11T19:41:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.280+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 shutdown_connections 2024-03-11T19:41:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.280+0000 7f1f23a00700 1 --2- 172.21.15.107:0/932075665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f1c108040 0x7f1f1c19b400 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.280+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 >> 172.21.15.107:0/932075665 conn(0x7f1f1c101ad0 msgr2=0x7f1f1c102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.281+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 shutdown_connections 2024-03-11T19:41:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:03.281+0000 7f1f23a00700 1 -- 172.21.15.107:0/932075665 wait complete. 2024-03-11T19:41:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (3/15)... 2024-03-11T19:41:04.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:03 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:03.970+0000 7f762bfad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:41:04.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: Activating manager daemon x 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: mgrmap e2: x(active, starting, since 0.00934995s) 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: Manager daemon x is now available 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:41:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' 2024-03-11T19:41:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' 2024-03-11T19:41:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:04 smithi107 ceph-mon[31712]: from='mgr.14100 172.21.15.107:0/2749502259' entity='mgr.x' 2024-03-11T19:41:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:05 smithi107 ceph-mon[31712]: mgrmap e3: x(active, since 1.01973s) 2024-03-11T19:41:07.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:06 smithi107 ceph-mon[31712]: mgrmap e4: x(active, since 2s) 2024-03-11T19:41:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:41:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:41:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:41:07.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:41:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:40:45.252069+0000", 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:41:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.701+0000 7ff2f3800700 1 Processor -- start 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.701+0000 7ff2f3800700 1 -- start start 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.701+0000 7ff2f3800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.701+0000 7ff2f3800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec07aed0 con 0x7ff2ec108040 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.702+0000 7ff2f0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.702+0000 7ff2f0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47862/0 (socket says 172.21.15.107:47862) 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.702+0000 7ff2f0e00700 1 -- 172.21.15.107:0/1634176948 learned_addr learned my addr 172.21.15.107:0/1634176948 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.702+0000 7ff2f0e00700 1 -- 172.21.15.107:0/1634176948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec10a970 con 0x7ff2ec108040 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.703+0000 7ff2f0e00700 1 --2- 172.21.15.107:0/1634176948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098a0 tx=0x7ff2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965b6aae3e0687b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.703+0000 7ff2eb400700 1 -- 172.21.15.107:0/1634176948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0004160 con 0x7ff2ec108040 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.703+0000 7ff2eb400700 1 -- 172.21.15.107:0/1634176948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e00042c0 con 0x7ff2ec108040 2024-03-11T19:41:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.703+0000 7ff2eb400700 1 -- 172.21.15.107:0/1634176948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0011ac0 con 0x7ff2ec108040 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 -- 172.21.15.107:0/1634176948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 msgr2=0x7ff2ec10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1634176948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098a0 tx=0x7ff2e0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 -- 172.21.15.107:0/1634176948 shutdown_connections 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1634176948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec10a430 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 -- 172.21.15.107:0/1634176948 >> 172.21.15.107:0/1634176948 conn(0x7ff2ec101ad0 msgr2=0x7ff2ec103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 -- 172.21.15.107:0/1634176948 shutdown_connections 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.704+0000 7ff2f3800700 1 -- 172.21.15.107:0/1634176948 wait complete. 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.705+0000 7ff2f3800700 1 Processor -- start 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f3800700 1 -- start start 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f3800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f3800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec107e00 con 0x7ff2ec108040 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47866/0 (socket says 172.21.15.107:47866) 2024-03-11T19:41:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.706+0000 7ff2f0e00700 1 -- 172.21.15.107:0/1730926413 learned_addr learned my addr 172.21.15.107:0/1730926413 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2f0e00700 1 -- 172.21.15.107:0/1730926413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009550 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2f0e00700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011a70 tx=0x7ff2e0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e00039c0 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e0003b20 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec105ff0 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0024610 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.707+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec106490 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.708+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7ff2e001a430 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.708+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec10b7d0 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.709+0000 7ff2e9600700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 0x7ff2d403b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.709+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2e004ccd0 con 0x7ff2ec108040 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.709+0000 7ff2ebe00700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 0x7ff2d403b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.710+0000 7ff2ebe00700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 0x7ff2d403b120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:06.715+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2e0010380 con 0x7ff2ec108040 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.014+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff2ec106aa0 con 0x7ff2ec108040 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.014+0000 7ff2e9600700 1 -- 172.21.15.107:0/1730926413 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff2e0018be0 con 0x7ff2ec108040 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 msgr2=0x7ff2d403b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 0x7ff2d403b120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d8006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 msgr2=0x7ff2ec1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011a70 tx=0x7ff2e0011b50 comp rx=0 tx=0).stop 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 shutdown_connections 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7ff2d4038c60 0x7ff2d403b120 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 --2- 172.21.15.107:0/1730926413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2ec108040 0x7ff2ec1078c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 >> 172.21.15.107:0/1730926413 conn(0x7ff2ec101ad0 msgr2=0x7ff2ec102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.017+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 shutdown_connections 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:07.018+0000 7ff2f3800700 1 -- 172.21.15.107:0/1730926413 wait complete. 2024-03-11T19:41:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-03-11T19:41:08.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:07 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1730926413' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:41:09.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:08 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2657153876' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-11T19:41:09.474 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:09.475 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:41:09.475 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:41:09.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:41:09.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:41:09.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:41:09.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:41:09.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.441+0000 7f1c35200700 1 Processor -- start 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.441+0000 7f1c35200700 1 -- start start 2024-03-11T19:41:09.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.441+0000 7f1c35200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:09.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.441+0000 7f1c35200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3010ead0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.442+0000 7f1c2de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:09.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.442+0000 7f1c2de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47878/0 (socket says 172.21.15.107:47878) 2024-03-11T19:41:09.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.442+0000 7f1c2de00700 1 -- 172.21.15.107:0/2370104315 learned_addr learned my addr 172.21.15.107:0/2370104315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:09.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.442+0000 7f1c2de00700 1 -- 172.21.15.107:0/2370104315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3010ee00 con 0x7f1c3010e0e0 2024-03-11T19:41:09.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.442+0000 7f1c2de00700 1 --2- 172.21.15.107:0/2370104315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c200098a0 tx=0x7f1c20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9f38ac9e074b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c2ca00700 1 -- 172.21.15.107:0/2370104315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c20004160 con 0x7f1c3010e0e0 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c2ca00700 1 -- 172.21.15.107:0/2370104315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c200042c0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c2ca00700 1 -- 172.21.15.107:0/2370104315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c20011ac0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 -- 172.21.15.107:0/2370104315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 msgr2=0x7f1c3010e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 --2- 172.21.15.107:0/2370104315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c200098a0 tx=0x7f1c20009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 -- 172.21.15.107:0/2370104315 shutdown_connections 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 --2- 172.21.15.107:0/2370104315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c3010e500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:09.483 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 -- 172.21.15.107:0/2370104315 >> 172.21.15.107:0/2370104315 conn(0x7f1c3007cdf0 msgr2=0x7f1c3007f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.443+0000 7f1c35200700 1 -- 172.21.15.107:0/2370104315 shutdown_connections 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.444+0000 7f1c35200700 1 -- 172.21.15.107:0/2370104315 wait complete. 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.444+0000 7f1c35200700 1 Processor -- start 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c35200700 1 -- start start 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c35200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c35200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301a1f30 con 0x7f1c3010e0e0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c2de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c2de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47892/0 (socket says 172.21.15.107:47892) 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.445+0000 7f1c2de00700 1 -- 172.21.15.107:0/2657153876 learned_addr learned my addr 172.21.15.107:0/2657153876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c2de00700 1 -- 172.21.15.107:0/2657153876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c20009550 con 0x7f1c3010e0e0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c2de00700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009fb0 tx=0x7f1c20011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c200038b0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c20003a10 con 0x7f1c3010e0e0 2024-03-11T19:41:09.484 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c301a2130 con 0x7f1c3010e0e0 2024-03-11T19:41:09.485 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c200195e0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.485 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.446+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c301a25d0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.485 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.447+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f1c2001a430 con 0x7f1c3010e0e0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.447+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c30112030 con 0x7f1c3010e0e0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.448+0000 7f1c26a00700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 0x7f1c1403b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.448+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c2004cd70 con 0x7f1c3010e0e0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.448+0000 7f1c2d400700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 0x7f1c1403b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.449+0000 7f1c2d400700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 0x7f1c1403b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006fd0 tx=0x7f1c18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.453+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c20003b80 con 0x7f1c3010e0e0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.668+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1c301a4eb0 con 0x7f1c3010e0e0 2024-03-11T19:41:09.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.670+0000 7f1c26a00700 1 -- 172.21.15.107:0/2657153876 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f1c20023850 con 0x7f1c3010e0e0 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.672+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 msgr2=0x7f1c1403b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.672+0000 7f1c35200700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 0x7f1c1403b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006fd0 tx=0x7f1c18006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.672+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 msgr2=0x7f1c301a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.672+0000 7f1c35200700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009fb0 tx=0x7f1c20011b20 comp rx=0 tx=0).stop 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 shutdown_connections 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f1c14038c60 0x7f1c1403b120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 --2- 172.21.15.107:0/2657153876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c3010e0e0 0x7f1c301a19f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 >> 172.21.15.107:0/2657153876 conn(0x7f1c3007cdf0 msgr2=0x7f1c3007e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 shutdown_connections 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:08.673+0000 7f1c35200700 1 -- 172.21.15.107:0/2657153876 wait complete. 2024-03-11T19:41:09.488 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-03-11T19:41:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:10 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1277590551' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-11T19:41:10.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:10 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setuser ceph since I am not root 2024-03-11T19:41:10.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:10 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setgroup ceph since I am not root 2024-03-11T19:41:11.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.101+0000 7fde22800700 1 Processor -- start 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.101+0000 7fde22800700 1 -- start start 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.102+0000 7fde22800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.102+0000 7fde22800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c107ff0 con 0x7fde1c107600 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.102+0000 7fde1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.102+0000 7fde1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47898/0 (socket says 172.21.15.107:47898) 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.102+0000 7fde1b400700 1 -- 172.21.15.107:0/90059787 learned_addr learned my addr 172.21.15.107:0/90059787 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:11.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.103+0000 7fde1b400700 1 -- 172.21.15.107:0/90059787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c1087b0 con 0x7fde1c107600 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.103+0000 7fde1b400700 1 --2- 172.21.15.107:0/90059787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0098a0 tx=0x7fde0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fde89a5f6f6511 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.103+0000 7fde1aa00700 1 -- 172.21.15.107:0/90059787 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde0c004160 con 0x7fde1c107600 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.103+0000 7fde1aa00700 1 -- 172.21.15.107:0/90059787 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde0c0042c0 con 0x7fde1c107600 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.103+0000 7fde1aa00700 1 -- 172.21.15.107:0/90059787 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde0c011ac0 con 0x7fde1c107600 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 -- 172.21.15.107:0/90059787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 msgr2=0x7fde1c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 --2- 172.21.15.107:0/90059787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0098a0 tx=0x7fde0c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:11.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 -- 172.21.15.107:0/90059787 shutdown_connections 2024-03-11T19:41:11.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 --2- 172.21.15.107:0/90059787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c107a20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:11.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 -- 172.21.15.107:0/90059787 >> 172.21.15.107:0/90059787 conn(0x7fde1c102c40 msgr2=0x7fde1c105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:11.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 -- 172.21.15.107:0/90059787 shutdown_connections 2024-03-11T19:41:11.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.104+0000 7fde22800700 1 -- 172.21.15.107:0/90059787 wait complete. 2024-03-11T19:41:11.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.105+0000 7fde22800700 1 Processor -- start 2024-03-11T19:41:11.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.105+0000 7fde22800700 1 -- start start 2024-03-11T19:41:11.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde22800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:11.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde22800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c19db20 con 0x7fde1c107600 2024-03-11T19:41:11.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:11.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47910/0 (socket says 172.21.15.107:47910) 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde1b400700 1 -- 172.21.15.107:0/1277590551 learned_addr learned my addr 172.21.15.107:0/1277590551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.106+0000 7fde1b400700 1 -- 172.21.15.107:0/1277590551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c009550 con 0x7fde1c107600 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde1b400700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000c00 tx=0x7fde0c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde0c003aa0 con 0x7fde1c107600 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde0c003c00 con 0x7fde1c107600 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c19dd20 con 0x7fde1c107600 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde0c0248a0 con 0x7fde1c107600 2024-03-11T19:41:11.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.107+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c19e1c0 con 0x7fde1c107600 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.108+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c10b9e0 con 0x7fde1c107600 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.109+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7fde0c01ac60 con 0x7fde1c107600 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.110+0000 7fde18c00700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 0x7fde0003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.110+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fde0c04ca00 con 0x7fde1c107600 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.110+0000 7fde13e00700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 0x7fde0003b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.110+0000 7fde13e00700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 0x7fde0003b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde04006fd0 tx=0x7fde04006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.115+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde0c018530 con 0x7fde1c107600 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.387+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fde1c10bbf0 con 0x7fde1c107600 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.441+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fde0c018350 con 0x7fde1c107600 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 msgr2=0x7fde0003b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde18c00700 1 -- 172.21.15.107:0/1277590551 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fde0c02d770 con 0x7fde1c107600 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 0x7fde0003b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde04006fd0 tx=0x7fde04006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 msgr2=0x7fde1c19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:11.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000c00 tx=0x7fde0c011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 shutdown_connections 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fde00038c60 0x7fde0003b120 secure :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde04006fd0 tx=0x7fde04006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 --2- 172.21.15.107:0/1277590551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1c107600 0x7fde1c19d5e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.444+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 >> 172.21.15.107:0/1277590551 conn(0x7fde1c102c40 msgr2=0x7fde1c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.446+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 shutdown_connections 2024-03-11T19:41:11.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:10.446+0000 7fde22800700 1 -- 172.21.15.107:0/1277590551 wait complete. 2024-03-11T19:41:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1277590551' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-11T19:41:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:11 smithi107 ceph-mon[31712]: mgrmap e5: x(active, since 6s) 2024-03-11T19:41:11.505 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:11 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:11.213+0000 7fb371d68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:41:11.505 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:11 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:11.339+0000 7fb371d68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:41:12.455 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:12.155+0000 7fb371d68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:41:12.455 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:12.278+0000 7fb371d68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:41:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3441822618' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:41:12.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:12.455+0000 7fb371d68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:41:12.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:12.582+0000 7fb371d68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:41:12.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:12.691+0000 7fb371d68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:41:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-11T19:41:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:41:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:41:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd4200700 1 Processor -- start 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd4200700 1 -- start start 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd4200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd4200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc107fb0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47940/0 (socket says 172.21.15.107:47940) 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.778+0000 7f0bd1800700 1 -- 172.21.15.107:0/4198704307 learned_addr learned my addr 172.21.15.107:0/4198704307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.779+0000 7f0bd1800700 1 -- 172.21.15.107:0/4198704307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc108770 con 0x7f0bcc1075c0 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.779+0000 7f0bd1800700 1 --2- 172.21.15.107:0/4198704307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688dc5dec541587 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.779+0000 7f0bcbe00700 1 -- 172.21.15.107:0/4198704307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0004160 con 0x7f0bcc1075c0 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bcbe00700 1 -- 172.21.15.107:0/4198704307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bc00042c0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bcbe00700 1 -- 172.21.15.107:0/4198704307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0011ac0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 -- 172.21.15.107:0/4198704307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 msgr2=0x7f0bcc1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 --2- 172.21.15.107:0/4198704307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 -- 172.21.15.107:0/4198704307 shutdown_connections 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 --2- 172.21.15.107:0/4198704307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc1079e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 -- 172.21.15.107:0/4198704307 >> 172.21.15.107:0/4198704307 conn(0x7f0bcc102be0 msgr2=0x7f0bcc105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 -- 172.21.15.107:0/4198704307 shutdown_connections 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.780+0000 7f0bd4200700 1 -- 172.21.15.107:0/4198704307 wait complete. 2024-03-11T19:41:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.781+0000 7f0bd4200700 1 Processor -- start 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.781+0000 7f0bd4200700 1 -- start start 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd4200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd4200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc19dad0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47956/0 (socket says 172.21.15.107:47956) 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd1800700 1 -- 172.21.15.107:0/3441822618 learned_addr learned my addr 172.21.15.107:0/3441822618 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.782+0000 7f0bd1800700 1 -- 172.21.15.107:0/3441822618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0009550 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bd1800700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0003f70 tx=0x7f0bc0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc00039f0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bc0003b50 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc19dcd0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc001a690 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.783+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc19e170 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.784+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f0bc0024430 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.784+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc10b9a0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.785+0000 7f0bca000700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 0x7f0bb403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.785+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0bc004cee0 con 0x7f0bcc1075c0 2024-03-11T19:41:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.785+0000 7f0bd0e00700 1 -- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 msgr2=0x7f0bb403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.785+0000 7f0bd0e00700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 0x7f0bb403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.790+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bc0018a40 con 0x7f0bcc1075c0 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.986+0000 7f0bd0e00700 1 -- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 msgr2=0x7f0bb403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:11.986+0000 7f0bd0e00700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 0x7f0bb403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.067+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0bcc1a0f40 con 0x7f0bcc1075c0 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.067+0000 7f0bca000700 1 -- 172.21.15.107:0/3441822618 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0bc0018350 con 0x7f0bcc1075c0 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 msgr2=0x7f0bb403b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 0x7f0bb403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 msgr2=0x7f0bcc19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0003f70 tx=0x7f0bc0011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 shutdown_connections 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7f0bb4038cb0 0x7f0bb403b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 --2- 172.21.15.107:0/3441822618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bcc1075c0 0x7f0bcc19d590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 >> 172.21.15.107:0/3441822618 conn(0x7f0bcc102be0 msgr2=0x7f0bcc1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 shutdown_connections 2024-03-11T19:41:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:12.070+0000 7f0bd4200700 1 -- 172.21.15.107:0/3441822618 wait complete. 2024-03-11T19:41:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-11T19:41:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-03-11T19:41:13.188 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:13.041+0000 7fb371d68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:41:13.455 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:13.183+0000 7fb371d68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:41:13.455 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:13.301+0000 7fb371d68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:41:13.456 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:13.427+0000 7fb371d68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:41:13.731 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:13.647+0000 7fb371d68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:41:15.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:14 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:14.911+0000 7fb371d68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:41:15.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:15.080+0000 7fb371d68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:41:15.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:15.495+0000 7fb371d68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:41:15.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:15.613+0000 7fb371d68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:41:16.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:15.867+0000 7fb371d68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:16.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:16.109+0000 7fb371d68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:41:16.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:16.479+0000 7fb371d68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:17.099 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:16.795+0000 7fb371d68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:41:17.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:17.094+0000 7fb371d68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:41:18.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:18.102+0000 7fb371d68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:41:19.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:18.788+0000 7fb371d68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:41:19.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:18.916+0000 7fb371d68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:41:19.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:19.196+0000 7fb371d68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:41:19.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: Active manager daemon x restarted 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: Activating manager daemon x 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: osdmap e2: 0 total, 0 up, 0 in 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: mgrmap e6: x(active, starting, since 0.00777849s) 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: Manager daemon x is now available 2024-03-11T19:41:19.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:19.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:19 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:20.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: Found migration_current of "None". Setting to last migration. 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:20.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:20 smithi107 ceph-mon[31712]: mgrmap e7: x(active, since 1.01636s) 2024-03-11T19:41:20.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:20.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-11T19:41:20.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:41:20.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.518+0000 7fde0a000700 1 Processor -- start 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.518+0000 7fde0a000700 1 -- start start 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.518+0000 7fde0a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.518+0000 7fde0a000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04108030 con 0x7fde04107640 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.519+0000 7fde03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:20.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.519+0000 7fde03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47972/0 (socket says 172.21.15.107:47972) 2024-03-11T19:41:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.519+0000 7fde03400700 1 -- 172.21.15.107:0/1501452135 learned_addr learned my addr 172.21.15.107:0/1501452135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.519+0000 7fde03400700 1 -- 172.21.15.107:0/1501452135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde041087f0 con 0x7fde04107640 2024-03-11T19:41:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.519+0000 7fde03400700 1 --2- 172.21.15.107:0/1501452135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddf40098a0 tx=0x7fddf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db35e2b5f09a1bf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.520+0000 7fde02000700 1 -- 172.21.15.107:0/1501452135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddf4004160 con 0x7fde04107640 2024-03-11T19:41:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.520+0000 7fde02000700 1 -- 172.21.15.107:0/1501452135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf40042c0 con 0x7fde04107640 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.520+0000 7fde02000700 1 -- 172.21.15.107:0/1501452135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddf4011ac0 con 0x7fde04107640 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.520+0000 7fde0a000700 1 -- 172.21.15.107:0/1501452135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 msgr2=0x7fde04107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.520+0000 7fde0a000700 1 --2- 172.21.15.107:0/1501452135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddf40098a0 tx=0x7fddf4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.521+0000 7fde0a000700 1 -- 172.21.15.107:0/1501452135 shutdown_connections 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.521+0000 7fde0a000700 1 --2- 172.21.15.107:0/1501452135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde04107a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.521+0000 7fde0a000700 1 -- 172.21.15.107:0/1501452135 >> 172.21.15.107:0/1501452135 conn(0x7fde04102ca0 msgr2=0x7fde041050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.521+0000 7fde0a000700 1 -- 172.21.15.107:0/1501452135 shutdown_connections 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.521+0000 7fde0a000700 1 -- 172.21.15.107:0/1501452135 wait complete. 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde0a000700 1 Processor -- start 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde0a000700 1 -- start start 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde0a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde0a000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0419dbd0 con 0x7fde04107640 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47988/0 (socket says 172.21.15.107:47988) 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.522+0000 7fde03400700 1 -- 172.21.15.107:0/2014035304 learned_addr learned my addr 172.21.15.107:0/2014035304 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fde03400700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009550 con 0x7fde04107640 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fde03400700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011780 tx=0x7fddf4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddf4003af0 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf4003c50 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddf40197e0 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.523+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0419ddd0 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.524+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0419e270 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fddf401a430 con 0x7fde04107640 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fddfbe00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdde803b880 con 0x7fdde8038cb0 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fddf404cfa0 con 0x7fde04107640 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fde02a00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.525+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.726+0000 7fde02a00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:13.726+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:14.127+0000 7fde02a00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:14.127+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:14.928+0000 7fde02a00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:14.928+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:16.530+0000 7fde02a00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/927061211 2024-03-11T19:41:20.999 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:16.530+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:19.205+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fddf4019a70 con 0x7fde04107640 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:19.206+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 msgr2=0x7fdde803b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:19.206+0000 7fddfbe00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.209+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7fddf404e610 con 0x7fde04107640 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.209+0000 7fddfbe00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 0x7fdde803e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.209+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdde803b880 con 0x7fdde803c5a0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.211+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 0x7fdde803e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.214+0000 7fde02a00700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 0x7fdde803e990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fddf0003a10 tx=0x7fddf0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.216+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdde803b880 con 0x7fdde803c5a0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.224+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fde0410b9f0 con 0x7fdde803c5a0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fddfbe00700 1 -- 172.21.15.107:0/2014035304 <== mgr.14120 v2:172.21.15.107:6800/3464148109 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fde0410b9f0 con 0x7fdde803c5a0 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 msgr2=0x7fdde803e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 0x7fdde803e990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fddf0003a10 tx=0x7fddf0009290 comp rx=0 tx=0).stop 2024-03-11T19:41:21.000 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 msgr2=0x7fde0419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fddf4011780 tx=0x7fddf4011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 shutdown_connections 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fdde803c5a0 0x7fdde803e990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde04107640 0x7fde0419d690 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 --2- 172.21.15.107:0/2014035304 >> [v2:172.21.15.107:6800/927061211,v1:172.21.15.107:6801/927061211] conn(0x7fdde8038cb0 0x7fdde803b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 >> 172.21.15.107:0/2014035304 conn(0x7fde04102ca0 msgr2=0x7fde04103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.225+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 shutdown_connections 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:20.226+0000 7fde0a000700 1 -- 172.21.15.107:0/2014035304 wait complete. 2024-03-11T19:41:21.001 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-03-11T19:41:21.002 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-03-11T19:41:21.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:21 smithi107 ceph-mon[31712]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:41:21.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:21 smithi107 ceph-mon[31712]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:41:21.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:21 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:22.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:20] ENGINE Bus STARTING 2024-03-11T19:41:22.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:21] ENGINE Serving on http://172.21.15.107:8765 2024-03-11T19:41:22.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:21] ENGINE Serving on https://172.21.15.107:7150 2024-03-11T19:41:22.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:21] ENGINE Bus STARTED 2024-03-11T19:41:22.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: mgrmap e8: x(active, since 2s) 2024-03-11T19:41:22.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:22.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:22 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:22.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.649+0000 7fbe3ee00700 1 Processor -- start 2024-03-11T19:41:22.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.650+0000 7fbe3ee00700 1 -- start start 2024-03-11T19:41:22.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.650+0000 7fbe3ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:22.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.650+0000 7fbe3ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3810a1f0 con 0x7fbe38109800 2024-03-11T19:41:22.587 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.650+0000 7fbe37e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.651+0000 7fbe37e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37462/0 (socket says 172.21.15.107:37462) 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.651+0000 7fbe37e00700 1 -- 172.21.15.107:0/2564811330 learned_addr learned my addr 172.21.15.107:0/2564811330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.651+0000 7fbe37e00700 1 -- 172.21.15.107:0/2564811330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3810a9b0 con 0x7fbe38109800 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.651+0000 7fbe37e00700 1 --2- 172.21.15.107:0/2564811330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe280098a0 tx=0x7fbe28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988049bfbad4f979 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.652+0000 7fbe36a00700 1 -- 172.21.15.107:0/2564811330 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe28004160 con 0x7fbe38109800 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.652+0000 7fbe36a00700 1 -- 172.21.15.107:0/2564811330 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe280042c0 con 0x7fbe38109800 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.652+0000 7fbe36a00700 1 -- 172.21.15.107:0/2564811330 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe28011ac0 con 0x7fbe38109800 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.652+0000 7fbe3ee00700 1 -- 172.21.15.107:0/2564811330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 msgr2=0x7fbe38109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.652+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/2564811330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe280098a0 tx=0x7fbe28009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.653+0000 7fbe3ee00700 1 -- 172.21.15.107:0/2564811330 shutdown_connections 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.653+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/2564811330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe38109c20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.653+0000 7fbe3ee00700 1 -- 172.21.15.107:0/2564811330 >> 172.21.15.107:0/2564811330 conn(0x7fbe3807cdf0 msgr2=0x7fbe381074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.653+0000 7fbe3ee00700 1 -- 172.21.15.107:0/2564811330 shutdown_connections 2024-03-11T19:41:22.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.653+0000 7fbe3ee00700 1 -- 172.21.15.107:0/2564811330 wait complete. 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe3ee00700 1 Processor -- start 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe3ee00700 1 -- start start 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe3ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe3ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3819db30 con 0x7fbe38109800 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe37e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe37e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37470/0 (socket says 172.21.15.107:37470) 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.654+0000 7fbe37e00700 1 -- 172.21.15.107:0/85989240 learned_addr learned my addr 172.21.15.107:0/85989240 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe37e00700 1 -- 172.21.15.107:0/85989240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009550 con 0x7fbe38109800 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe37e00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe28011780 tx=0x7fbe28011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe28003af0 con 0x7fbe38109800 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe28003c50 con 0x7fbe38109800 2024-03-11T19:41:22.589 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3819dd30 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe280197e0 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.655+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3819e1d0 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.656+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3810dbe0 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.656+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fbe2801aba0 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.657+0000 7fbe34c00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 0x7fbe1c03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.657+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe2804d000 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.657+0000 7fbe37400700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 0x7fbe1c03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.658+0000 7fbe37400700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 0x7fbe1c03b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe24006fd0 tx=0x7fbe24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.662+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe2801a430 con 0x7fbe38109800 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.897+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe38107400 con 0x7fbe1c038cb0 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.907+0000 7fbe34c00700 1 -- 172.21.15.107:0/85989240 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbe38107400 con 0x7fbe1c038cb0 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 msgr2=0x7fbe1c03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:22.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 0x7fbe1c03b170 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe24006fd0 tx=0x7fbe24006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 msgr2=0x7fbe3819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe28011780 tx=0x7fbe28011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 shutdown_connections 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fbe1c038cb0 0x7fbe1c03b170 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 --2- 172.21.15.107:0/85989240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe38109800 0x7fbe3819d5f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 >> 172.21.15.107:0/85989240 conn(0x7fbe3807cdf0 msgr2=0x7fbe38106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 shutdown_connections 2024-03-11T19:41:22.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:21.910+0000 7fbe3ee00700 1 -- 172.21.15.107:0/85989240 wait complete. 2024-03-11T19:41:23.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:23 smithi107 ceph-mon[31712]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:24.156 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.178+0000 7f967f000700 1 Processor -- start 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.178+0000 7f967f000700 1 -- start start 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.179+0000 7f967f000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.179+0000 7f967f000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967807b840 con 0x7f967807ce50 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.179+0000 7f9677e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.179+0000 7f9677e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37474/0 (socket says 172.21.15.107:37474) 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.179+0000 7f9677e00700 1 -- 172.21.15.107:0/3973047547 learned_addr learned my addr 172.21.15.107:0/3973047547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.180+0000 7f9677e00700 1 -- 172.21.15.107:0/3973047547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967807b980 con 0x7f967807ce50 2024-03-11T19:41:24.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.180+0000 7f9677e00700 1 --2- 172.21.15.107:0/3973047547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96680098a0 tx=0x7f9668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4f5489b3347e50 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.180+0000 7f9676a00700 1 -- 172.21.15.107:0/3973047547 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9668004160 con 0x7f967807ce50 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.180+0000 7f9676a00700 1 -- 172.21.15.107:0/3973047547 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96680042c0 con 0x7f967807ce50 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.180+0000 7f9676a00700 1 -- 172.21.15.107:0/3973047547 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9668011ac0 con 0x7f967807ce50 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 -- 172.21.15.107:0/3973047547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 msgr2=0x7f967807b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 --2- 172.21.15.107:0/3973047547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96680098a0 tx=0x7f9668009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:24.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 -- 172.21.15.107:0/3973047547 shutdown_connections 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 --2- 172.21.15.107:0/3973047547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967807b300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 -- 172.21.15.107:0/3973047547 >> 172.21.15.107:0/3973047547 conn(0x7f9678102be0 msgr2=0x7f9678105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.181+0000 7f967f000700 1 -- 172.21.15.107:0/3973047547 shutdown_connections 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.182+0000 7f967f000700 1 -- 172.21.15.107:0/3973047547 wait complete. 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.182+0000 7f967f000700 1 Processor -- start 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f967f000700 1 -- start start 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f967f000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f967f000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967819daa0 con 0x7f967807ce50 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f9677e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f9677e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37482/0 (socket says 172.21.15.107:37482) 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.183+0000 7f9677e00700 1 -- 172.21.15.107:0/3297381421 learned_addr learned my addr 172.21.15.107:0/3297381421 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f9677e00700 1 -- 172.21.15.107:0/3297381421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9668009550 con 0x7f967807ce50 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f9677e00700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9668009130 tx=0x7f9668011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:24.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9668003aa0 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9668003c00 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9668024800 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.184+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967819dca0 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.185+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967819e140 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.186+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f966801abd0 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.186+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967810d950 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.186+0000 7f9674c00700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 0x7f965c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.186+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f966804ceb0 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.187+0000 7f9677400700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 0x7f965c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.187+0000 7f9677400700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 0x7f965c03b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.192+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9668024a70 con 0x7f967807ce50 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.414+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9678103e60 con 0x7f965c038d00 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.415+0000 7f9674c00700 1 -- 172.21.15.107:0/3297381421 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9678103e60 con 0x7f965c038d00 2024-03-11T19:41:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 msgr2=0x7f965c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 0x7f965c03b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 msgr2=0x7f967819d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9668009130 tx=0x7f9668011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 shutdown_connections 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f965c038d00 0x7f965c03b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 --2- 172.21.15.107:0/3297381421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f967807ce50 0x7f967819d560 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 >> 172.21.15.107:0/3297381421 conn(0x7f9678102be0 msgr2=0x7f9678103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 shutdown_connections 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:23.418+0000 7f967f000700 1 -- 172.21.15.107:0/3297381421 wait complete. 2024-03-11T19:41:24.161 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-03-11T19:41:24.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:24 smithi107 ceph-mon[31712]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: Generating public/private rsa key pair. 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: Your identification has been saved in /tmp/tmpi9y5leq8/key. 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: Your public key has been saved in /tmp/tmpi9y5leq8/key.pub. 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: The key fingerprint is: 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: SHA256:Xczieh73rcu03Rs6Nte1ktkxJpeI6CqaKumMrEfQii8 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: The key's randomart image is: 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: +---[RSA 3072]----+ 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: | | 2024-03-11T19:41:25.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: | o | 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: | . . + | 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: |. . o o | 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: |.o S + . . .| 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: |o . o . o *.| 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: | + o o . O.*| 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: |E o .. + o @.B+| 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: |*B.o. ... . ..@+=| 2024-03-11T19:41:25.297 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: +----[SHA256]-----+ 2024-03-11T19:41:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.720+0000 7f3e0e400700 1 Processor -- start 2024-03-11T19:41:25.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.720+0000 7f3e0e400700 1 -- start start 2024-03-11T19:41:25.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.721+0000 7f3e0e400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.721+0000 7f3e0e400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0810a1f0 con 0x7f3e08109800 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.721+0000 7f3e07400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.721+0000 7f3e07400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37494/0 (socket says 172.21.15.107:37494) 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.721+0000 7f3e07400700 1 -- 172.21.15.107:0/185175877 learned_addr learned my addr 172.21.15.107:0/185175877 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.722+0000 7f3e07400700 1 -- 172.21.15.107:0/185175877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0810a9b0 con 0x7f3e08109800 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.722+0000 7f3e07400700 1 --2- 172.21.15.107:0/185175877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098a0 tx=0x7f3df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82de645b21a5e516 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.722+0000 7f3e06a00700 1 -- 172.21.15.107:0/185175877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df8004160 con 0x7f3e08109800 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.722+0000 7f3e06a00700 1 -- 172.21.15.107:0/185175877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df80042c0 con 0x7f3e08109800 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.722+0000 7f3e06a00700 1 -- 172.21.15.107:0/185175877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df8011ac0 con 0x7f3e08109800 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 -- 172.21.15.107:0/185175877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 msgr2=0x7f3e08109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 --2- 172.21.15.107:0/185175877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3df80098a0 tx=0x7f3df8009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:25.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 -- 172.21.15.107:0/185175877 shutdown_connections 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 --2- 172.21.15.107:0/185175877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e08109c20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 -- 172.21.15.107:0/185175877 >> 172.21.15.107:0/185175877 conn(0x7f3e0807cdf0 msgr2=0x7f3e081074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 -- 172.21.15.107:0/185175877 shutdown_connections 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.723+0000 7f3e0e400700 1 -- 172.21.15.107:0/185175877 wait complete. 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.724+0000 7f3e0e400700 1 Processor -- start 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.724+0000 7f3e0e400700 1 -- start start 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e0e400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e0e400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0819db30 con 0x7f3e08109800 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e07400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e07400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37508/0 (socket says 172.21.15.107:37508) 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e07400700 1 -- 172.21.15.107:0/2964523126 learned_addr learned my addr 172.21.15.107:0/2964523126 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.725+0000 7f3e07400700 1 -- 172.21.15.107:0/2964523126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df8009550 con 0x7f3e08109800 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e07400700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011780 tx=0x7f3df8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df8003af0 con 0x7f3e08109800 2024-03-11T19:41:25.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df8003c50 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0819dd30 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df80197e0 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.726+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0819e1d0 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.727+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f3df801a430 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.727+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0810dbe0 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.727+0000 7f3e04c00700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 0x7f3dec03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.727+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3df804d020 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.728+0000 7f3dffe00700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 0x7f3dec03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.728+0000 7f3dffe00700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 0x7f3dec03b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006fd0 tx=0x7f3df0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.733+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3df802da40 con 0x7f3e08109800 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:24.953+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e081073e0 con 0x7f3dec038d00 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.061+0000 7f3e04c00700 1 -- 172.21.15.107:0/2964523126 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3e081073e0 con 0x7f3dec038d00 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 msgr2=0x7f3dec03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:25.796 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 0x7f3dec03b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006fd0 tx=0x7f3df0006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 msgr2=0x7f3e0819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3df8011780 tx=0x7f3df8011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 shutdown_connections 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f3dec038d00 0x7f3dec03b1c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 --2- 172.21.15.107:0/2964523126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e08109800 0x7f3e0819d5f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 >> 172.21.15.107:0/2964523126 conn(0x7f3e0807cdf0 msgr2=0x7f3e08106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 shutdown_connections 2024-03-11T19:41:25.797 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:25.064+0000 7f3e0e400700 1 -- 172.21.15.107:0/2964523126 wait complete. 2024-03-11T19:41:26.312 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:26 smithi107 ceph-mon[31712]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:26.312 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:26 smithi107 ceph-mon[31712]: Generating ssh key... 2024-03-11T19:41:26.312 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:26 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:26.312 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:26 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.354+0000 7f4982a00700 1 Processor -- start 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.354+0000 7f4982a00700 1 -- start start 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f4982a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f4982a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c07b820 con 0x7f497c07ce30 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f497be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f497be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45268/0 (socket says 172.21.15.107:45268) 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f497be00700 1 -- 172.21.15.107:0/1167747623 learned_addr learned my addr 172.21.15.107:0/1167747623 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.355+0000 7f497be00700 1 -- 172.21.15.107:0/1167747623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c07b960 con 0x7f497c07ce30 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.356+0000 7f497be00700 1 --2- 172.21.15.107:0/1167747623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f496c0098a0 tx=0x7f496c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d27d6c5309063 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:27.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.356+0000 7f497b400700 1 -- 172.21.15.107:0/1167747623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496c004160 con 0x7f497c07ce30 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.356+0000 7f497b400700 1 -- 172.21.15.107:0/1167747623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f496c0042c0 con 0x7f497c07ce30 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.356+0000 7f497b400700 1 -- 172.21.15.107:0/1167747623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496c011ac0 con 0x7f497c07ce30 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 -- 172.21.15.107:0/1167747623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 msgr2=0x7f497c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 --2- 172.21.15.107:0/1167747623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f496c0098a0 tx=0x7f496c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 -- 172.21.15.107:0/1167747623 shutdown_connections 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 --2- 172.21.15.107:0/1167747623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c07b2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 -- 172.21.15.107:0/1167747623 >> 172.21.15.107:0/1167747623 conn(0x7f497c102be0 msgr2=0x7f497c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 -- 172.21.15.107:0/1167747623 shutdown_connections 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.357+0000 7f4982a00700 1 -- 172.21.15.107:0/1167747623 wait complete. 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.358+0000 7f4982a00700 1 Processor -- start 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.358+0000 7f4982a00700 1 -- start start 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.358+0000 7f4982a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:27.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.358+0000 7f4982a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c19da90 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.359+0000 7f497be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.359+0000 7f497be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45280/0 (socket says 172.21.15.107:45280) 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.359+0000 7f497be00700 1 -- 172.21.15.107:0/2803394184 learned_addr learned my addr 172.21.15.107:0/2803394184 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.359+0000 7f497be00700 1 -- 172.21.15.107:0/2803394184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009550 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.359+0000 7f497be00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f496c009130 tx=0x7f496c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.360+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496c003aa0 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.360+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f496c003c00 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.360+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c19dc90 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.360+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496c024800 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.360+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c19e130 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.361+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c10d970 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.361+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f496c01abd0 con 0x7f497c07ce30 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.361+0000 7f4979600700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 0x7f496003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.362+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f496c04ce80 con 0x7f497c07ce30 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.362+0000 7f4973e00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 0x7f496003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.362+0000 7f4973e00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 0x7f496003b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4964006fd0 tx=0x7f4964006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.367+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f496c018760 con 0x7f497c07ce30 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.588+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f497c103ec0 con 0x7f4960038d00 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.590+0000 7f4979600700 1 -- 172.21.15.107:0/2803394184 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f497c103ec0 con 0x7f4960038d00 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 msgr2=0x7f496003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 0x7f496003b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4964006fd0 tx=0x7f4964006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 msgr2=0x7f497c19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f496c009130 tx=0x7f496c011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 shutdown_connections 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f4960038d00 0x7f496003b1c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.592+0000 7f4982a00700 1 --2- 172.21.15.107:0/2803394184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f497c07ce30 0x7f497c19d550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.593+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 >> 172.21.15.107:0/2803394184 conn(0x7f497c102be0 msgr2=0x7f497c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.593+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 shutdown_connections 2024-03-11T19:41:27.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:26.593+0000 7f4982a00700 1 -- 172.21.15.107:0/2803394184 wait complete. 2024-03-11T19:41:27.365 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:41:27.365 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-03-11T19:41:27.365 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-03-11T19:41:27.892 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:27 smithi107 ceph-mon[31712]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:28.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:28 smithi107 ceph-mon[31712]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:31.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:30 smithi107 ceph-mon[31712]: Deploying cephadm binary to smithi107 2024-03-11T19:41:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-03-11T19:41:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.972+0000 7fc3c3200700 1 Processor -- start 2024-03-11T19:41:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.972+0000 7fc3c3200700 1 -- start start 2024-03-11T19:41:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.973+0000 7fc3c3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:35.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.973+0000 7fc3c3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc07ae30 con 0x7fc3bc1080e0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.973+0000 7fc3bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.973+0000 7fc3bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45288/0 (socket says 172.21.15.107:45288) 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.973+0000 7fc3bbe00700 1 -- 172.21.15.107:0/3692596801 learned_addr learned my addr 172.21.15.107:0/3692596801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.974+0000 7fc3bbe00700 1 -- 172.21.15.107:0/3692596801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc07af70 con 0x7fc3bc1080e0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.974+0000 7fc3bbe00700 1 --2- 172.21.15.107:0/3692596801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009e80 tx=0x7fc3ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13bde663e39d98 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.974+0000 7fc3baa00700 1 -- 172.21.15.107:0/3692596801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac00e070 con 0x7fc3bc1080e0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.974+0000 7fc3baa00700 1 -- 172.21.15.107:0/3692596801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3ac00b8c0 con 0x7fc3bc1080e0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.974+0000 7fc3baa00700 1 -- 172.21.15.107:0/3692596801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac004550 con 0x7fc3bc1080e0 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 -- 172.21.15.107:0/3692596801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 msgr2=0x7fc3bc10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3692596801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009e80 tx=0x7fc3ac00b330 comp rx=0 tx=0).stop 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 -- 172.21.15.107:0/3692596801 shutdown_connections 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3692596801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc10a4d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 -- 172.21.15.107:0/3692596801 >> 172.21.15.107:0/3692596801 conn(0x7fc3bc101ad0 msgr2=0x7fc3bc103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 -- 172.21.15.107:0/3692596801 shutdown_connections 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.975+0000 7fc3c3200700 1 -- 172.21.15.107:0/3692596801 wait complete. 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.976+0000 7fc3c3200700 1 Processor -- start 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.976+0000 7fc3c3200700 1 -- start start 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3c3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3c3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc19b870 con 0x7fc3bc1080e0 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45294/0 (socket says 172.21.15.107:45294) 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3bbe00700 1 -- 172.21.15.107:0/3196937705 learned_addr learned my addr 172.21.15.107:0/3196937705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.977+0000 7fc3bbe00700 1 -- 172.21.15.107:0/3196937705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac009b30 con 0x7fc3bc1080e0 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3bbe00700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac00e040 con 0x7fc3bc1080e0 2024-03-11T19:41:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3ac0128f0 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3ac01c770 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc19ba70 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.978+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc19bef0 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.979+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc10b7a0 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.979+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fc3ac012a60 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.980+0000 7fc3b8c00700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 0x7fc3a003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.980+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc3ac04cd00 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.980+0000 7fc3bb400700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 0x7fc3a003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.981+0000 7fc3bb400700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 0x7fc3a003b1c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006fd0 tx=0x7fc3a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:27.985+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3ac053080 con 0x7fc3bc1080e0 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:28.209+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3bc102cd0 con 0x7fc3a0038d00 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.492+0000 7fc3b8c00700 1 -- 172.21.15.107:0/3196937705 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc3bc102cd0 con 0x7fc3a0038d00 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.496+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 msgr2=0x7fc3a003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.496+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 0x7fc3a003b1c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006fd0 tx=0x7fc3a8006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.496+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 msgr2=0x7fc3bc19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.496+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac003d00 comp rx=0 tx=0).stop 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 shutdown_connections 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc3a0038d00 0x7fc3a003b1c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 --2- 172.21.15.107:0/3196937705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc1080e0 0x7fc3bc19b330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 >> 172.21.15.107:0/3196937705 conn(0x7fc3bc101ad0 msgr2=0x7fc3bc1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 shutdown_connections 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:34.498+0000 7fc3c3200700 1 -- 172.21.15.107:0/3196937705 wait complete. 2024-03-11T19:41:35.394 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-03-11T19:41:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:35 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:35 smithi107 ceph-mon[31712]: Added host smithi107 2024-03-11T19:41:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:35 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:37.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:37 smithi107 ceph-mon[31712]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:37.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:37 smithi107 ceph-mon[31712]: Saving service mon spec with placement count:5 2024-03-11T19:41:37.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:37 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:38.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-11T19:41:38.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.057+0000 7f05d1c00700 1 Processor -- start 2024-03-11T19:41:38.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.057+0000 7f05d1c00700 1 -- start start 2024-03-11T19:41:38.135 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.058+0000 7f05d1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.058+0000 7f05d1c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc07ae50 con 0x7f05cc105cf0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.058+0000 7f05cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.058+0000 7f05cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54944/0 (socket says 172.21.15.107:54944) 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.058+0000 7f05cbe00700 1 -- 172.21.15.107:0/3749863185 learned_addr learned my addr 172.21.15.107:0/3749863185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.059+0000 7f05cbe00700 1 -- 172.21.15.107:0/3749863185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc106650 con 0x7f05cc105cf0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.059+0000 7f05cbe00700 1 --2- 172.21.15.107:0/3749863185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098a0 tx=0x7f05bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2885ce7a2f56ca21 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.059+0000 7f05caa00700 1 -- 172.21.15.107:0/3749863185 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05bc004160 con 0x7f05cc105cf0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.059+0000 7f05caa00700 1 -- 172.21.15.107:0/3749863185 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05bc0042c0 con 0x7f05cc105cf0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.059+0000 7f05caa00700 1 -- 172.21.15.107:0/3749863185 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05bc011ac0 con 0x7f05cc105cf0 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 -- 172.21.15.107:0/3749863185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 msgr2=0x7f05cc106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 --2- 172.21.15.107:0/3749863185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098a0 tx=0x7f05bc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 -- 172.21.15.107:0/3749863185 shutdown_connections 2024-03-11T19:41:38.136 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 --2- 172.21.15.107:0/3749863185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc106110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 -- 172.21.15.107:0/3749863185 >> 172.21.15.107:0/3749863185 conn(0x7f05cc101970 msgr2=0x7f05cc103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 -- 172.21.15.107:0/3749863185 shutdown_connections 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.060+0000 7f05d1c00700 1 -- 172.21.15.107:0/3749863185 wait complete. 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.061+0000 7f05d1c00700 1 Processor -- start 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05d1c00700 1 -- start start 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05d1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05d1c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc19d8f0 con 0x7f05cc105cf0 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54946/0 (socket says 172.21.15.107:54946) 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.062+0000 7f05cbe00700 1 -- 172.21.15.107:0/2242780197 learned_addr learned my addr 172.21.15.107:0/2242780197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05cbe00700 1 -- 172.21.15.107:0/2242780197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009550 con 0x7f05cc105cf0 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05cbe00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009130 tx=0x7f05bc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05bc003aa0 con 0x7f05cc105cf0 2024-03-11T19:41:38.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05bc003c00 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05bc024800 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc19daf0 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.063+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc19df90 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.064+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f05bc01abd0 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.064+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc197020 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.065+0000 7f05c8c00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 0x7f05b003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.065+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f05bc04ce80 con 0x7f05cc105cf0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.065+0000 7f05cb400700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 0x7f05b003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.066+0000 7f05cb400700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 0x7f05b003b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05b8006fd0 tx=0x7f05b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:38.140 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.070+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05bc018760 con 0x7f05cc105cf0 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.326+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f05cc02da30 con 0x7f05b0038d00 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.333+0000 7f05c8c00700 1 -- 172.21.15.107:0/2242780197 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f05cc02da30 con 0x7f05b0038d00 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.336+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 msgr2=0x7f05b003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.336+0000 7f05d1c00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 0x7f05b003b1c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05b8006fd0 tx=0x7f05b8006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.336+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 msgr2=0x7f05cc19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.336+0000 7f05d1c00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009130 tx=0x7f05bc011b40 comp rx=0 tx=0).stop 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 shutdown_connections 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f05b0038d00 0x7f05b003b1c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 --2- 172.21.15.107:0/2242780197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc105cf0 0x7f05cc19d3b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 >> 172.21.15.107:0/2242780197 conn(0x7f05cc101970 msgr2=0x7f05cc194160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 shutdown_connections 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:36.337+0000 7f05d1c00700 1 -- 172.21.15.107:0/2242780197 wait complete. 2024-03-11T19:41:38.141 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-03-11T19:41:38.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:38 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:38.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:38 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f329d200700 1 Processor -- start 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f329d200700 1 -- start start 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f329d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f329d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298107e00 con 0x7f3298107410 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f3297400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f3297400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54958/0 (socket says 172.21.15.107:54958) 2024-03-11T19:41:39.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.736+0000 7f3297400700 1 -- 172.21.15.107:0/1840181017 learned_addr learned my addr 172.21.15.107:0/1840181017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:39.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.737+0000 7f3297400700 1 -- 172.21.15.107:0/1840181017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32981085c0 con 0x7f3298107410 2024-03-11T19:41:39.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.737+0000 7f3297400700 1 --2- 172.21.15.107:0/1840181017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32880098a0 tx=0x7f3288009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88050d9be76f1a29 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:39.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.737+0000 7f3296000700 1 -- 172.21.15.107:0/1840181017 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3288004160 con 0x7f3298107410 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.737+0000 7f3296000700 1 -- 172.21.15.107:0/1840181017 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32880042c0 con 0x7f3298107410 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f3296000700 1 -- 172.21.15.107:0/1840181017 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3288011a80 con 0x7f3298107410 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f329d200700 1 -- 172.21.15.107:0/1840181017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 msgr2=0x7f3298107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f329d200700 1 --2- 172.21.15.107:0/1840181017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32880098a0 tx=0x7f3288009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f329d200700 1 -- 172.21.15.107:0/1840181017 shutdown_connections 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f329d200700 1 --2- 172.21.15.107:0/1840181017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f3298107830 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.738+0000 7f329d200700 1 -- 172.21.15.107:0/1840181017 >> 172.21.15.107:0/1840181017 conn(0x7f3298102a90 msgr2=0x7f3298104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.739+0000 7f329d200700 1 -- 172.21.15.107:0/1840181017 shutdown_connections 2024-03-11T19:41:39.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.739+0000 7f329d200700 1 -- 172.21.15.107:0/1840181017 wait complete. 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f329d200700 1 Processor -- start 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f329d200700 1 -- start start 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f329d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f329d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329807d0d0 con 0x7f3298107410 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f3297400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f3297400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54964/0 (socket says 172.21.15.107:54964) 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.740+0000 7f3297400700 1 -- 172.21.15.107:0/1532953259 learned_addr learned my addr 172.21.15.107:0/1532953259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f3297400700 1 -- 172.21.15.107:0/1532953259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288009550 con 0x7f3298107410 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f3297400700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3288011740 tx=0x7f3288011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32880039e0 con 0x7f3298107410 2024-03-11T19:41:39.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3288003b40 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3288024740 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329807b2c0 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.741+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329807b760 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.743+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f328801ab30 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.743+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329810b7f0 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.743+0000 7f328fe00700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 0x7f327c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.743+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f328804ce60 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.743+0000 7f3296a00700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 0x7f327c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.744+0000 7f3296a00700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 0x7f327c03b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3284006fd0 tx=0x7f3284006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.749+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3288010380 con 0x7f3298107410 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.982+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f329807bd70 con 0x7f327c038d00 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.990+0000 7f328fe00700 1 -- 172.21.15.107:0/1532953259 <== mgr.14120 v2:172.21.15.107:6800/3464148109 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f329807bd70 con 0x7f327c038d00 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 msgr2=0x7f327c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:39.735 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 0x7f327c03b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3284006fd0 tx=0x7f3284006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 msgr2=0x7f329807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3288011740 tx=0x7f3288011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 shutdown_connections 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f327c038d00 0x7f327c03b1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 --2- 172.21.15.107:0/1532953259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298107410 0x7f329807cb90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.992+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 >> 172.21.15.107:0/1532953259 conn(0x7f3298102a90 msgr2=0x7f3298103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.993+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 shutdown_connections 2024-03-11T19:41:39.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:38.993+0000 7f329d200700 1 -- 172.21.15.107:0/1532953259 wait complete. 2024-03-11T19:41:40.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:39 smithi107 ceph-mon[31712]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:40.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:39 smithi107 ceph-mon[31712]: Saving service mgr spec with placement count:2 2024-03-11T19:41:40.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:39 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:40.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:39 smithi107 ceph-mon[31712]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:41:41.909 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:41 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4105959695' entity='client.admin' 2024-03-11T19:41:41.909 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:41 smithi107 ceph-mon[31712]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:41:41.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.339+0000 7f22dfe00700 1 Processor -- start 2024-03-11T19:41:41.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.339+0000 7f22dfe00700 1 -- start start 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dfe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d807b930 con 0x7f22d807ceb0 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54966/0 (socket says 172.21.15.107:54966) 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dd400700 1 -- 172.21.15.107:0/3694695944 learned_addr learned my addr 172.21.15.107:0/3694695944 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.340+0000 7f22dd400700 1 -- 172.21.15.107:0/3694695944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d807ba70 con 0x7f22d807ceb0 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22dd400700 1 --2- 172.21.15.107:0/3694695944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0098a0 tx=0x7f22cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a7ebebe844fe84 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22d7e00700 1 -- 172.21.15.107:0/3694695944 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22cc004160 con 0x7f22d807ceb0 2024-03-11T19:41:41.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22d7e00700 1 -- 172.21.15.107:0/3694695944 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22cc0042c0 con 0x7f22d807ceb0 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22d7e00700 1 -- 172.21.15.107:0/3694695944 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22cc011a80 con 0x7f22d807ceb0 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22dfe00700 1 -- 172.21.15.107:0/3694695944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 msgr2=0x7f22d807b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.341+0000 7f22dfe00700 1 --2- 172.21.15.107:0/3694695944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0098a0 tx=0x7f22cc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.342+0000 7f22dfe00700 1 -- 172.21.15.107:0/3694695944 shutdown_connections 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.342+0000 7f22dfe00700 1 --2- 172.21.15.107:0/3694695944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d807b360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.342+0000 7f22dfe00700 1 -- 172.21.15.107:0/3694695944 >> 172.21.15.107:0/3694695944 conn(0x7f22d8102c40 msgr2=0x7f22d8105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.342+0000 7f22dfe00700 1 -- 172.21.15.107:0/3694695944 shutdown_connections 2024-03-11T19:41:41.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.342+0000 7f22dfe00700 1 -- 172.21.15.107:0/3694695944 wait complete. 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.343+0000 7f22dfe00700 1 Processor -- start 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.343+0000 7f22dfe00700 1 -- start start 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.343+0000 7f22dfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.343+0000 7f22dfe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81a1e10 con 0x7f22d807ceb0 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54968/0 (socket says 172.21.15.107:54968) 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22dd400700 1 -- 172.21.15.107:0/4105959695 learned_addr learned my addr 172.21.15.107:0/4105959695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22dd400700 1 -- 172.21.15.107:0/4105959695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc009550 con 0x7f22d807ceb0 2024-03-11T19:41:41.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22dd400700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22cc011740 tx=0x7f22cc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.344+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22cc003b20 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.345+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22cc01a910 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.345+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22cc0248c0 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.345+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d81a2010 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.345+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d81a24b0 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.346+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f22cc01aa80 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.346+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d819b790 con 0x7f22d807ceb0 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.346+0000 7f22d6000700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 0x7f22c003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:41.926 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.346+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f22cc04d010 con 0x7f22d807ceb0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.347+0000 7f22dca00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 0x7f22c003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.347+0000 7f22dca00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 0x7f22c003b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.352+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22cc01adf0 con 0x7f22d807ceb0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.576+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f22d802d8e0 con 0x7f22d807ceb0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.581+0000 7f22d6000700 1 -- 172.21.15.107:0/4105959695 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f22cc018be0 con 0x7f22d807ceb0 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.583+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 msgr2=0x7f22c003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.583+0000 7f22dfe00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 0x7f22c003b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.583+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 msgr2=0x7f22d81a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:41.927 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.583+0000 7f22dfe00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22cc011740 tx=0x7f22cc011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 shutdown_connections 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f22c0038d00 0x7f22c003b1c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 --2- 172.21.15.107:0/4105959695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d807ceb0 0x7f22d81a18d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 >> 172.21.15.107:0/4105959695 conn(0x7f22d8102c40 msgr2=0x7f22d81988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 shutdown_connections 2024-03-11T19:41:41.928 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:40.584+0000 7f22dfe00700 1 -- 172.21.15.107:0/4105959695 wait complete. 2024-03-11T19:41:43.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:43 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:43.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:43 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1320890862' entity='client.admin' 2024-03-11T19:41:43.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.563+0000 7efcd7c00700 1 Processor -- start 2024-03-11T19:41:43.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.563+0000 7efcd7c00700 1 -- start start 2024-03-11T19:41:43.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.564+0000 7efcd7c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.564+0000 7efcd7c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0107fb0 con 0x7efcd01075c0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.564+0000 7efcd5200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.564+0000 7efcd5200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54980/0 (socket says 172.21.15.107:54980) 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.564+0000 7efcd5200700 1 -- 172.21.15.107:0/780489688 learned_addr learned my addr 172.21.15.107:0/780489688 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.565+0000 7efcd5200700 1 -- 172.21.15.107:0/780489688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0108770 con 0x7efcd01075c0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.565+0000 7efcd5200700 1 --2- 172.21.15.107:0/780489688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcc40098a0 tx=0x7efcc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1713fbfb65b8e286 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.565+0000 7efccf400700 1 -- 172.21.15.107:0/780489688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcc4004160 con 0x7efcd01075c0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.565+0000 7efccf400700 1 -- 172.21.15.107:0/780489688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcc40042c0 con 0x7efcd01075c0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.565+0000 7efccf400700 1 -- 172.21.15.107:0/780489688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcc4011a80 con 0x7efcd01075c0 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 -- 172.21.15.107:0/780489688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 msgr2=0x7efcd01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 --2- 172.21.15.107:0/780489688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcc40098a0 tx=0x7efcc4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 -- 172.21.15.107:0/780489688 shutdown_connections 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 --2- 172.21.15.107:0/780489688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd01079e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 -- 172.21.15.107:0/780489688 >> 172.21.15.107:0/780489688 conn(0x7efcd0102be0 msgr2=0x7efcd0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:43.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 -- 172.21.15.107:0/780489688 shutdown_connections 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.566+0000 7efcd7c00700 1 -- 172.21.15.107:0/780489688 wait complete. 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.567+0000 7efcd7c00700 1 Processor -- start 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd7c00700 1 -- start start 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd7c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd7c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd007d0d0 con 0x7efcd01075c0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd5200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd5200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54982/0 (socket says 172.21.15.107:54982) 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.568+0000 7efcd5200700 1 -- 172.21.15.107:0/1320890862 learned_addr learned my addr 172.21.15.107:0/1320890862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efcd5200700 1 -- 172.21.15.107:0/1320890862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc4009550 con 0x7efcd01075c0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efcd5200700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcc4011740 tx=0x7efcc4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcc40039e0 con 0x7efcd01075c0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcc4003b40 con 0x7efcd01075c0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd007b2c0 con 0x7efcd01075c0 2024-03-11T19:41:43.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcc4024740 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.569+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd007b760 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.570+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd010b9a0 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.570+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7efcc401a430 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.571+0000 7efccd600700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 0x7efcb803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.571+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efcc404ceb0 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.571+0000 7efccfe00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 0x7efcb803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.572+0000 7efccfe00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 0x7efcb803b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcbc006fd0 tx=0x7efcbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.577+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efcc4010380 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.803+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7efcd010bbb0 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.807+0000 7efccd600700 1 -- 172.21.15.107:0/1320890862 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7efcc4018be0 con 0x7efcd01075c0 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 msgr2=0x7efcb803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 0x7efcb803b1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcbc006fd0 tx=0x7efcbc006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 msgr2=0x7efcd007cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcc4011740 tx=0x7efcc4011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:43.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 shutdown_connections 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7efcb8038d00 0x7efcb803b1c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.812+0000 7efcd7c00700 1 --2- 172.21.15.107:0/1320890862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcd01075c0 0x7efcd007cb90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.813+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 >> 172.21.15.107:0/1320890862 conn(0x7efcd0102be0 msgr2=0x7efcd01036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.813+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 shutdown_connections 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:42.813+0000 7efcd7c00700 1 -- 172.21.15.107:0/1320890862 wait complete. 2024-03-11T19:41:43.526 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-03-11T19:41:44.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:44 smithi107 ceph-mon[31712]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:41:45.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:45 smithi107 ceph-mon[31712]: from='mgr.14120 172.21.15.107:0/1245624503' entity='mgr.x' 2024-03-11T19:41:45.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:45 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2279943041' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-11T19:41:45.749 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setuser ceph since I am not root 2024-03-11T19:41:45.749 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setgroup ceph since I am not root 2024-03-11T19:41:46.534 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:46.076+0000 7f0ac73dd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:41:46.534 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:46.198+0000 7f0ac73dd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:41:46.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:46 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2279943041' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-11T19:41:46.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:46 smithi107 ceph-mon[31712]: mgrmap e9: x(active, since 26s) 2024-03-11T19:41:46.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.168+0000 7faaaca00700 1 Processor -- start 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.168+0000 7faaaca00700 1 -- start start 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.169+0000 7faaaca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.169+0000 7faaaca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8107d90 con 0x7faaa8107430 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.169+0000 7faaa6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.169+0000 7faaa6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54988/0 (socket says 172.21.15.107:54988) 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.169+0000 7faaa6800700 1 -- 172.21.15.107:0/1808085730 learned_addr learned my addr 172.21.15.107:0/1808085730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.170+0000 7faaa6800700 1 -- 172.21.15.107:0/1808085730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8107ed0 con 0x7faaa8107430 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.170+0000 7faaa6800700 1 --2- 172.21.15.107:0/1808085730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa980098a0 tx=0x7faa98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b629c7200b4a8e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.170+0000 7faaa5400700 1 -- 172.21.15.107:0/1808085730 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa98004160 con 0x7faaa8107430 2024-03-11T19:41:46.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaa5400700 1 -- 172.21.15.107:0/1808085730 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa980042c0 con 0x7faaa8107430 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaa5400700 1 -- 172.21.15.107:0/1808085730 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa98011a80 con 0x7faaa8107430 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaaca00700 1 -- 172.21.15.107:0/1808085730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 msgr2=0x7faaa8107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaaca00700 1 --2- 172.21.15.107:0/1808085730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa980098a0 tx=0x7faa98009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaaca00700 1 -- 172.21.15.107:0/1808085730 shutdown_connections 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaaca00700 1 --2- 172.21.15.107:0/1808085730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa8107850 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.171+0000 7faaaca00700 1 -- 172.21.15.107:0/1808085730 >> 172.21.15.107:0/1808085730 conn(0x7faaa8102b10 msgr2=0x7faaa8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.172+0000 7faaaca00700 1 -- 172.21.15.107:0/1808085730 shutdown_connections 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.172+0000 7faaaca00700 1 -- 172.21.15.107:0/1808085730 wait complete. 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.173+0000 7faaaca00700 1 Processor -- start 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.173+0000 7faaaca00700 1 -- start start 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaaca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaaca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa819b630 con 0x7faaa8107430 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaa6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaa6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54992/0 (socket says 172.21.15.107:54992) 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaa6800700 1 -- 172.21.15.107:0/2279943041 learned_addr learned my addr 172.21.15.107:0/2279943041 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.174+0000 7faaa6800700 1 -- 172.21.15.107:0/2279943041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98009550 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faaa6800700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa98011740 tx=0x7faa98011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa98003b20 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa9801a910 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa980248c0 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa819b830 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.175+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa819bcd0 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.176+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7faa9801aa80 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.176+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa81092f0 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.177+0000 7faa9f400700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c038d00 0x7faa8c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.177+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faa9804d010 con 0x7faaa8107430 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.177+0000 7faaa5e00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c038d00 0x7faa8c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.178+0000 7faaa5e00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c038d00 0x7faa8c03b1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa94006fd0 tx=0x7faa94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.182+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa9801adf0 con 0x7faaa8107430 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:44.464+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7faaa819e610 con 0x7faaa8107430 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.264+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7faa98018be0 con 0x7faaa8107430 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c038d00 msgr2=0x7faa8c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faaaca00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c038d00 0x7faa8c03b1c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa94006fd0 tx=0x7faa94006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 msgr2=0x7faaa819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faaaca00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa98011740 tx=0x7faa98011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7faa9802d910 con 0x7faaa8107430 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.268+0000 7faa9f400700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c03ca70 0x7faa8c03ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 shutdown_connections 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faaaca00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c03ca70 0x7faa8c03ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faaaca00700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa8107430 0x7faaa819b0f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 >> 172.21.15.107:0/2279943041 conn(0x7faaa8102b10 msgr2=0x7faaa8103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faa9f400700 1 -- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c03ca70 msgr2=0x7faa8c03ee60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-11T19:41:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faa9f400700 1 --2- 172.21.15.107:0/2279943041 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7faa8c03ca70 0x7faa8c03ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.270+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 shutdown_connections 2024-03-11T19:41:46.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:45.271+0000 7faaaca00700 1 -- 172.21.15.107:0/2279943041 wait complete. 2024-03-11T19:41:47.270 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.001+0000 7f0ac73dd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:41:47.270 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.125+0000 7f0ac73dd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:41:47.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.302+0000 7f0ac73dd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:41:47.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.427+0000 7f0ac73dd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:41:47.898 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.546+0000 7f0ac73dd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:41:48.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:48 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4196988677' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:41:48.171 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:47.894+0000 7f0ac73dd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:41:48.171 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:48.044+0000 7f0ac73dd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:41:48.171 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:48.167+0000 7f0ac73dd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:41:48.428 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:48.292+0000 7f0ac73dd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.731+0000 7fc5f1000700 1 Processor -- start 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.731+0000 7fc5f1000700 1 -- start start 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.731+0000 7fc5f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.731+0000 7fc5f1000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec07aed0 con 0x7fc5ec108040 2024-03-11T19:41:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.732+0000 7fc5e9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.732+0000 7fc5e9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41356/0 (socket says 172.21.15.107:41356) 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.732+0000 7fc5e9e00700 1 -- 172.21.15.107:0/2673826408 learned_addr learned my addr 172.21.15.107:0/2673826408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.732+0000 7fc5e9e00700 1 -- 172.21.15.107:0/2673826408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec10a970 con 0x7fc5ec108040 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.733+0000 7fc5e9e00700 1 --2- 172.21.15.107:0/2673826408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0098a0 tx=0x7fc5dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd20f96dcb48ff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.733+0000 7fc5e8a00700 1 -- 172.21.15.107:0/2673826408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5dc004160 con 0x7fc5ec108040 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.733+0000 7fc5e8a00700 1 -- 172.21.15.107:0/2673826408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc0042c0 con 0x7fc5ec108040 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.733+0000 7fc5e8a00700 1 -- 172.21.15.107:0/2673826408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5dc011a80 con 0x7fc5ec108040 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 -- 172.21.15.107:0/2673826408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 msgr2=0x7fc5ec10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 --2- 172.21.15.107:0/2673826408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0098a0 tx=0x7fc5dc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 -- 172.21.15.107:0/2673826408 shutdown_connections 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 --2- 172.21.15.107:0/2673826408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec10a430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 -- 172.21.15.107:0/2673826408 >> 172.21.15.107:0/2673826408 conn(0x7fc5ec101ad0 msgr2=0x7fc5ec103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 -- 172.21.15.107:0/2673826408 shutdown_connections 2024-03-11T19:41:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.734+0000 7fc5f1000700 1 -- 172.21.15.107:0/2673826408 wait complete. 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.735+0000 7fc5f1000700 1 Processor -- start 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.735+0000 7fc5f1000700 1 -- start start 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.735+0000 7fc5f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.735+0000 7fc5f1000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19b8d0 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.736+0000 7fc5e9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.736+0000 7fc5e9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41362/0 (socket says 172.21.15.107:41362) 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.736+0000 7fc5e9e00700 1 -- 172.21.15.107:0/4196988677 learned_addr learned my addr 172.21.15.107:0/4196988677 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.736+0000 7fc5e9e00700 1 -- 172.21.15.107:0/4196988677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc009550 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.736+0000 7fc5e9e00700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc003f20 tx=0x7fc5dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.737+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5dc0039f0 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.737+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc003b50 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.737+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec19bad0 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.737+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5dc01a6d0 con 0x7fc5ec108040 2024-03-11T19:41:48.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.737+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec19bf70 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.738+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fc5dc01a940 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.738+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec10b810 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.738+0000 7fc5e2a00700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 0x7fc5d003b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.738+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc5dc04d1d0 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.739+0000 7fc5e9400700 1 -- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 msgr2=0x7fc5d003b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.739+0000 7fc5e9400700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 0x7fc5d003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.744+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc5dc02c350 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.940+0000 7fc5e9400700 1 -- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 msgr2=0x7fc5d003b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:47.940+0000 7fc5e9400700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 0x7fc5d003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.020+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc5ec19e850 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.021+0000 7fc5e2a00700 1 -- 172.21.15.107:0/4196988677 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc5dc02c350 con 0x7fc5ec108040 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 msgr2=0x7fc5d003b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 0x7fc5d003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 msgr2=0x7fc5ec19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc003f20 tx=0x7fc5dc011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 shutdown_connections 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7fc5d0038d10 0x7fc5d003b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 --2- 172.21.15.107:0/4196988677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec108040 0x7fc5ec19b390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 >> 172.21.15.107:0/4196988677 conn(0x7fc5ec101ad0 msgr2=0x7fc5ec102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 shutdown_connections 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:48.024+0000 7fc5f1000700 1 -- 172.21.15.107:0/4196988677 wait complete. 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-11T19:41:48.692 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-03-11T19:41:48.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:48.526+0000 7f0ac73dd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:41:50.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:49.809+0000 7f0ac73dd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:41:50.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:49.978+0000 7f0ac73dd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:41:50.662 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:50.391+0000 7f0ac73dd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:41:50.662 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:50.508+0000 7f0ac73dd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:41:51.008 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:50.762+0000 7f0ac73dd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:51.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:51.003+0000 7f0ac73dd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:41:51.693 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:51.374+0000 7f0ac73dd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:41:51.693 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:51.688+0000 7f0ac73dd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:41:52.264 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:51.991+0000 7f0ac73dd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:41:53.528 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:53.055+0000 7f0ac73dd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:41:53.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:53.761+0000 7f0ac73dd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:41:54.177 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:53.891+0000 7f0ac73dd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:41:54.171+0000 7f0ac73dd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: Active manager daemon x restarted 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: Activating manager daemon x 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: osdmap e3: 0 total, 0 up, 0 in 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: mgrmap e10: x(active, starting, since 0.00912006s) 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:41:54.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:41:54.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:41:54.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:41:54.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:54 smithi107 ceph-mon[31712]: Manager daemon x is now available 2024-03-11T19:41:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:41:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:41:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:55 smithi107 ceph-mon[31712]: mgrmap e11: x(active, since 1.0165s) 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:54] ENGINE Bus STARTING 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:55] ENGINE Serving on https://172.21.15.107:7150 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:55] ENGINE Serving on http://172.21.15.107:8765 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: [11/Mar/2024:19:41:55] ENGINE Bus STARTED 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:41:56.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:56 smithi107 ceph-mon[31712]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:41:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:41:56.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.247+0000 7f176be00700 1 Processor -- start 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.247+0000 7f176be00700 1 -- start start 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.247+0000 7f176be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.248+0000 7f176be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c07d1b0 con 0x7f176c07cd90 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.248+0000 7f176aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.248+0000 7f176aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41370/0 (socket says 172.21.15.107:41370) 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.248+0000 7f176aa00700 1 -- 172.21.15.107:0/4282957099 learned_addr learned my addr 172.21.15.107:0/4282957099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.248+0000 7f176aa00700 1 -- 172.21.15.107:0/4282957099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c07b840 con 0x7f176c07cd90 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.249+0000 7f176aa00700 1 --2- 172.21.15.107:0/4282957099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f175c0098a0 tx=0x7f175c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0ab291607a6bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.249+0000 7f1769600700 1 -- 172.21.15.107:0/4282957099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f175c004160 con 0x7f176c07cd90 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.249+0000 7f1769600700 1 -- 172.21.15.107:0/4282957099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175c0042c0 con 0x7f176c07cd90 2024-03-11T19:41:56.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.249+0000 7f1769600700 1 -- 172.21.15.107:0/4282957099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f175c011a80 con 0x7f176c07cd90 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 -- 172.21.15.107:0/4282957099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 msgr2=0x7f176c07b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 --2- 172.21.15.107:0/4282957099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f175c0098a0 tx=0x7f175c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 -- 172.21.15.107:0/4282957099 shutdown_connections 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 --2- 172.21.15.107:0/4282957099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c07b300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 -- 172.21.15.107:0/4282957099 >> 172.21.15.107:0/4282957099 conn(0x7f176c102b10 msgr2=0x7f176c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 -- 172.21.15.107:0/4282957099 shutdown_connections 2024-03-11T19:41:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.250+0000 7f176be00700 1 -- 172.21.15.107:0/4282957099 wait complete. 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.251+0000 7f176be00700 1 Processor -- start 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.251+0000 7f176be00700 1 -- start start 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c19d830 con 0x7f176c07cd90 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41374/0 (socket says 172.21.15.107:41374) 2024-03-11T19:41:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176aa00700 1 -- 172.21.15.107:0/962421790 learned_addr learned my addr 172.21.15.107:0/962421790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176aa00700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009550 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.252+0000 7f176aa00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f175c011740 tx=0x7f175c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.253+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f175c003b20 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.253+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f175c01a910 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.253+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c19da30 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.253+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f175c0248c0 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.253+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c19ded0 con 0x7f176c07cd90 2024-03-11T19:41:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.254+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f175c01aa80 con 0x7f176c07cd90 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.255+0000 7f1763400700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.255+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f175003b8d0 con 0x7f1750038d00 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.255+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f175c04d190 con 0x7f176c07cd90 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.255+0000 7f176a000700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.255+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.456+0000 7f176a000700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:56.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.456+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.857+0000 7f176a000700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:49.857+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:50.657+0000 7f176a000700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:50.657+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:52.259+0000 7f176a000700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3464148109 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:52.259+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:41:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:54.179+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f175c01a430 con 0x7f176c07cd90 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:54.180+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 msgr2=0x7f175003b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:54.180+0000 7f1763400700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.184+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7f175c02bb60 con 0x7f176c07cd90 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.184+0000 7f1763400700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 0x7f175003eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.184+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f175003b8d0 con 0x7f175003c780 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.186+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 0x7f175003eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:56.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.188+0000 7f176a000700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 0x7f175003eb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1758003a10 tx=0x7f17580092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.189+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f175003b8d0 con 0x7f175003c780 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.196+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f176c10d760 con 0x7f175003c780 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f1763400700 1 -- 172.21.15.107:0/962421790 <== mgr.14152 v2:172.21.15.107:6800/1546798510 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f176c10d760 con 0x7f175003c780 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 msgr2=0x7f175003eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 0x7f175003eb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1758003a10 tx=0x7f17580092b0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 msgr2=0x7f176c19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f175c011740 tx=0x7f175c011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:56.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 shutdown_connections 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f175003c780 0x7f175003eb70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c07cd90 0x7f176c19d2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 --2- 172.21.15.107:0/962421790 >> [v2:172.21.15.107:6800/3464148109,v1:172.21.15.107:6801/3464148109] conn(0x7f1750038d00 0x7f175003b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.197+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 >> 172.21.15.107:0/962421790 conn(0x7f176c102b10 msgr2=0x7f176c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.198+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 shutdown_connections 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:55.198+0000 7f176be00700 1 -- 172.21.15.107:0/962421790 wait complete. 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-03-11T19:41:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-03-11T19:41:57.328 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:57 smithi107 ceph-mon[31712]: mgrmap e12: x(active, since 2s) 2024-03-11T19:41:57.328 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.484+0000 7ffb7e200700 1 Processor -- start 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.484+0000 7ffb7e200700 1 -- start start 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.484+0000 7ffb7e200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.485+0000 7ffb7e200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7807b8f0 con 0x7ffb7807ce70 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.485+0000 7ffb77400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.485+0000 7ffb77400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33644/0 (socket says 172.21.15.107:33644) 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.485+0000 7ffb77400700 1 -- 172.21.15.107:0/2931834903 learned_addr learned my addr 172.21.15.107:0/2931834903 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.485+0000 7ffb77400700 1 -- 172.21.15.107:0/2931834903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7807ba30 con 0x7ffb7807ce70 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.486+0000 7ffb77400700 1 --2- 172.21.15.107:0/2931834903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb680098a0 tx=0x7ffb68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c07da359eef1404 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:58.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.486+0000 7ffb76000700 1 -- 172.21.15.107:0/2931834903 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb68004160 con 0x7ffb7807ce70 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.486+0000 7ffb76000700 1 -- 172.21.15.107:0/2931834903 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb680042c0 con 0x7ffb7807ce70 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.486+0000 7ffb76000700 1 -- 172.21.15.107:0/2931834903 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb68011a80 con 0x7ffb7807ce70 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 -- 172.21.15.107:0/2931834903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 msgr2=0x7ffb7807b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2931834903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb680098a0 tx=0x7ffb68009bb0 comp rx=0 tx=0).stop 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 -- 172.21.15.107:0/2931834903 shutdown_connections 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2931834903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7807b320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 -- 172.21.15.107:0/2931834903 >> 172.21.15.107:0/2931834903 conn(0x7ffb78102be0 msgr2=0x7ffb78105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 -- 172.21.15.107:0/2931834903 shutdown_connections 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.487+0000 7ffb7e200700 1 -- 172.21.15.107:0/2931834903 wait complete. 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.488+0000 7ffb7e200700 1 Processor -- start 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.488+0000 7ffb7e200700 1 -- start start 2024-03-11T19:41:58.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb7e200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb7e200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7819d9f0 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb77400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb77400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33658/0 (socket says 172.21.15.107:33658) 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb77400700 1 -- 172.21.15.107:0/2656744424 learned_addr learned my addr 172.21.15.107:0/2656744424 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.489+0000 7ffb77400700 1 -- 172.21.15.107:0/2656744424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb68009550 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb77400700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb68011740 tx=0x7ffb68011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb68003b20 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb6801a910 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb680248c0 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7819dbf0 con 0x7ffb7807ce70 2024-03-11T19:41:58.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.490+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7819e090 con 0x7ffb7807ce70 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.492+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7ffb6801a430 con 0x7ffb7807ce70 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.492+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7810d9e0 con 0x7ffb7807ce70 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.492+0000 7ffb6fe00700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 0x7ffb5c03b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.492+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb6804d1f0 con 0x7ffb7807ce70 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.492+0000 7ffb76a00700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 0x7ffb5c03b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.493+0000 7ffb76a00700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 0x7ffb5c03b220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb64006fd0 tx=0x7ffb64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.500+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb68010380 con 0x7ffb7807ce70 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:57.723+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb78103d80 con 0x7ffb5c038d60 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.151+0000 7ffb6fe00700 1 -- 172.21.15.107:0/2656744424 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ffb78103d80 con 0x7ffb5c038d60 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 msgr2=0x7ffb5c03b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 0x7ffb5c03b220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb64006fd0 tx=0x7ffb64006e40 comp rx=0 tx=0).stop 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 msgr2=0x7ffb7819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb68011740 tx=0x7ffb68011b00 comp rx=0 tx=0).stop 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 shutdown_connections 2024-03-11T19:41:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ffb5c038d60 0x7ffb5c03b220 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 --2- 172.21.15.107:0/2656744424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb7807ce70 0x7ffb7819d4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 >> 172.21.15.107:0/2656744424 conn(0x7ffb78102be0 msgr2=0x7ffb78103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.154+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 shutdown_connections 2024-03-11T19:41:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:58.155+0000 7ffb7e200700 1 -- 172.21.15.107:0/2656744424 wait complete. 2024-03-11T19:41:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-03-11T19:41:59.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:59 smithi107 ceph-mon[31712]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:41:59.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:41:59.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:41:59.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:41:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:01.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:01 smithi107 ceph-mon[31712]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:42:01.196 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:01 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:01.211 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$YR1MPQZLf4TazNDrFkUEouumSK/o4NKAsudDrpgwmI9maMJZixUzG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710186119, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.507+0000 7f35e1e00700 1 Processor -- start 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.507+0000 7f35e1e00700 1 -- start start 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.508+0000 7f35e1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.508+0000 7f35e1e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc107fc0 con 0x7f35dc1075d0 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.508+0000 7f35daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.508+0000 7f35daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33674/0 (socket says 172.21.15.107:33674) 2024-03-11T19:42:01.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.508+0000 7f35daa00700 1 -- 172.21.15.107:0/1136623878 learned_addr learned my addr 172.21.15.107:0/1136623878 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:01.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.509+0000 7f35daa00700 1 -- 172.21.15.107:0/1136623878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc108780 con 0x7f35dc1075d0 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.509+0000 7f35daa00700 1 --2- 172.21.15.107:0/1136623878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0098a0 tx=0x7f35cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ab1dc361b404f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.509+0000 7f35d9600700 1 -- 172.21.15.107:0/1136623878 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35cc004160 con 0x7f35dc1075d0 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.509+0000 7f35d9600700 1 -- 172.21.15.107:0/1136623878 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35cc0042c0 con 0x7f35dc1075d0 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.509+0000 7f35d9600700 1 -- 172.21.15.107:0/1136623878 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35cc011a80 con 0x7f35dc1075d0 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 -- 172.21.15.107:0/1136623878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 msgr2=0x7f35dc1079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1136623878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0098a0 tx=0x7f35cc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 -- 172.21.15.107:0/1136623878 shutdown_connections 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1136623878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc1079f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 -- 172.21.15.107:0/1136623878 >> 172.21.15.107:0/1136623878 conn(0x7f35dc102c10 msgr2=0x7f35dc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.510+0000 7f35e1e00700 1 -- 172.21.15.107:0/1136623878 shutdown_connections 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.511+0000 7f35e1e00700 1 -- 172.21.15.107:0/1136623878 wait complete. 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.511+0000 7f35e1e00700 1 Processor -- start 2024-03-11T19:42:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35e1e00700 1 -- start start 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35e1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35e1e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc07d0d0 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33678/0 (socket says 172.21.15.107:33678) 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.512+0000 7f35daa00700 1 -- 172.21.15.107:0/1566365559 learned_addr learned my addr 172.21.15.107:0/1566365559 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35daa00700 1 -- 172.21.15.107:0/1566365559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009550 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35daa00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35cc011740 tx=0x7f35cc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35cc0039e0 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc07b2c0 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35cc003b40 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35cc024740 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.513+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc07b760 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.514+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f35cc01a430 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.515+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc10b9b0 con 0x7f35dc1075d0 2024-03-11T19:42:01.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.515+0000 7f35e0a00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 0x7f35c003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:01.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.515+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35cc04d130 con 0x7f35dc1075d0 2024-03-11T19:42:01.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.515+0000 7f35da000700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 0x7f35c003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:01.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.517+0000 7f35da000700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 0x7f35c003b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:01.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.525+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35cc010380 con 0x7f35dc1075d0 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:41:59.772+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f35dc0630c0 con 0x7f35c0038d00 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.000+0000 7f35e0a00700 1 -- 172.21.15.107:0/1566365559 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f35dc0630c0 con 0x7f35c0038d00 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 msgr2=0x7f35c003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 0x7f35c003b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 msgr2=0x7f35dc07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35cc011740 tx=0x7f35cc011b00 comp rx=0 tx=0).stop 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 shutdown_connections 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f35c0038d00 0x7f35c003b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 --2- 172.21.15.107:0/1566365559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35dc1075d0 0x7f35dc07cb90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 >> 172.21.15.107:0/1566365559 conn(0x7f35dc102c10 msgr2=0x7f35dc103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:01.220 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 shutdown_connections 2024-03-11T19:42:01.221 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:00.003+0000 7f35e1e00700 1 -- 172.21.15.107:0/1566365559 wait complete. 2024-03-11T19:42:01.221 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-03-11T19:42:02.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:02 smithi107 ceph-mon[31712]: mgrmap e13: x(active, since 6s) 2024-03-11T19:42:03.247 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/203319184' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-11T19:42:03.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-03-11T19:42:03.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.286+0000 7fa804a00700 1 Processor -- start 2024-03-11T19:42:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.286+0000 7fa804a00700 1 -- start start 2024-03-11T19:42:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.286+0000 7fa804a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.286+0000 7fa804a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80010a1d0 con 0x7fa8001097e0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.287+0000 7fa7fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.287+0000 7fa7fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33686/0 (socket says 172.21.15.107:33686) 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.287+0000 7fa7fde00700 1 -- 172.21.15.107:0/3226757705 learned_addr learned my addr 172.21.15.107:0/3226757705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.287+0000 7fa7fde00700 1 -- 172.21.15.107:0/3226757705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80010a990 con 0x7fa8001097e0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.288+0000 7fa7fde00700 1 --2- 172.21.15.107:0/3226757705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00098a0 tx=0x7fa7f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed04ad48e80a375c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.288+0000 7fa7fca00700 1 -- 172.21.15.107:0/3226757705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f0004160 con 0x7fa8001097e0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.288+0000 7fa7fca00700 1 -- 172.21.15.107:0/3226757705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f00042c0 con 0x7fa8001097e0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.288+0000 7fa7fca00700 1 -- 172.21.15.107:0/3226757705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f0011a80 con 0x7fa8001097e0 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 -- 172.21.15.107:0/3226757705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 msgr2=0x7fa800109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 --2- 172.21.15.107:0/3226757705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00098a0 tx=0x7fa7f0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 -- 172.21.15.107:0/3226757705 shutdown_connections 2024-03-11T19:42:03.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 --2- 172.21.15.107:0/3226757705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa800109c00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:03.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 -- 172.21.15.107:0/3226757705 >> 172.21.15.107:0/3226757705 conn(0x7fa80007cc30 msgr2=0x7fa8001074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:03.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 -- 172.21.15.107:0/3226757705 shutdown_connections 2024-03-11T19:42:03.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.289+0000 7fa804a00700 1 -- 172.21.15.107:0/3226757705 wait complete. 2024-03-11T19:42:03.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.290+0000 7fa804a00700 1 Processor -- start 2024-03-11T19:42:03.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.290+0000 7fa804a00700 1 -- start start 2024-03-11T19:42:03.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa804a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:03.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa804a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80019dad0 con 0x7fa8001097e0 2024-03-11T19:42:03.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa7fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:03.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa7fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33690/0 (socket says 172.21.15.107:33690) 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa7fde00700 1 -- 172.21.15.107:0/203319184 learned_addr learned my addr 172.21.15.107:0/203319184 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.291+0000 7fa7fde00700 1 -- 172.21.15.107:0/203319184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0009550 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa7fde00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0011740 tx=0x7fa7f0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f00039e0 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80019dcd0 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7f0003b40 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f0024740 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.292+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80019e170 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.293+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80010dbc0 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.294+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fa7f001ab30 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.294+0000 7fa7f6a00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 0x7fa7e403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.294+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7f004d040 con 0x7fa8001097e0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.294+0000 7fa7fd400700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 0x7fa7e403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.295+0000 7fa7fd400700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 0x7fa7e403b2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.300+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7f0010380 con 0x7fa8001097e0 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.516+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa8001a0ab0 con 0x7fa8001097e0 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.517+0000 7fa7f6a00700 1 -- 172.21.15.107:0/203319184 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa7f0018be0 con 0x7fa8001097e0 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 msgr2=0x7fa7e403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 0x7fa7e403b2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 msgr2=0x7fa80019d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0011740 tx=0x7fa7f0011b00 comp rx=0 tx=0).stop 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 shutdown_connections 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7e4038e00 0x7fa7e403b2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 --2- 172.21.15.107:0/203319184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8001097e0 0x7fa80019d590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.520+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 >> 172.21.15.107:0/203319184 conn(0x7fa80007cc30 msgr2=0x7fa800106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.521+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 shutdown_connections 2024-03-11T19:42:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:02.521+0000 7fa804a00700 1 -- 172.21.15.107:0/203319184 wait complete. 2024-03-11T19:42:03.301 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:42:03.301 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-11T19:42:03.327 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:42:03.327 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-11T19:42:03.327 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-11T19:42:03.327 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-11T19:42:03.334 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-03-11T19:42:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-03-11T19:42:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-03-11T19:42:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: Password: jjt9luxadk 2024-03-11T19:42:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:03.336 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config directory 2024-03-11T19:42:03.338 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-03-11T19:42:04.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.962+0000 7fd531c00700 1 Processor -- start 2024-03-11T19:42:04.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.962+0000 7fd531c00700 1 -- start start 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.962+0000 7fd531c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.962+0000 7fd531c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c10a1f0 con 0x7fd52c109800 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.963+0000 7fd52aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.963+0000 7fd52aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33704/0 (socket says 172.21.15.107:33704) 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.963+0000 7fd52aa00700 1 -- 172.21.15.107:0/2622054144 learned_addr learned my addr 172.21.15.107:0/2622054144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.963+0000 7fd52aa00700 1 -- 172.21.15.107:0/2622054144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c10a9b0 con 0x7fd52c109800 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.964+0000 7fd52aa00700 1 --2- 172.21.15.107:0/2622054144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0098a0 tx=0x7fd51c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5727ff00a96a68 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.964+0000 7fd52a000700 1 -- 172.21.15.107:0/2622054144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c004160 con 0x7fd52c109800 2024-03-11T19:42:04.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.964+0000 7fd52a000700 1 -- 172.21.15.107:0/2622054144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd51c0042c0 con 0x7fd52c109800 2024-03-11T19:42:04.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.964+0000 7fd52a000700 1 -- 172.21.15.107:0/2622054144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c011a80 con 0x7fd52c109800 2024-03-11T19:42:04.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 -- 172.21.15.107:0/2622054144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 msgr2=0x7fd52c109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 --2- 172.21.15.107:0/2622054144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0098a0 tx=0x7fd51c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 -- 172.21.15.107:0/2622054144 shutdown_connections 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 --2- 172.21.15.107:0/2622054144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c109c20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 -- 172.21.15.107:0/2622054144 >> 172.21.15.107:0/2622054144 conn(0x7fd52c07cdf0 msgr2=0x7fd52c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 -- 172.21.15.107:0/2622054144 shutdown_connections 2024-03-11T19:42:04.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.965+0000 7fd531c00700 1 -- 172.21.15.107:0/2622054144 wait complete. 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.966+0000 7fd531c00700 1 Processor -- start 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.966+0000 7fd531c00700 1 -- start start 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.966+0000 7fd531c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.966+0000 7fd531c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c19daf0 con 0x7fd52c109800 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd52aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd52aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33720/0 (socket says 172.21.15.107:33720) 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd52aa00700 1 -- 172.21.15.107:0/3723497906 learned_addr learned my addr 172.21.15.107:0/3723497906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd52aa00700 1 -- 172.21.15.107:0/3723497906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009550 con 0x7fd52c109800 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd52aa00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000c00 tx=0x7fd51c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c0039e0 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.967+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd51c003b40 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.968+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c19dcf0 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.968+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c024820 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.968+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c19e190 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.969+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fd51c01ab80 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.969+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c10dbe0 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.969+0000 7fd523400700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 0x7fd51003b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.969+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd51c04cf60 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.970+0000 7fd523e00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 0x7fd51003b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.970+0000 7fd523e00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 0x7fd51003b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:03.974+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd51c01a950 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.191+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd52c1a0ad0 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.192+0000 7fd523400700 1 -- 172.21.15.107:0/3723497906 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd51c04c7f0 con 0x7fd52c109800 2024-03-11T19:42:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.194+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 msgr2=0x7fd51003b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.194+0000 7fd531c00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 0x7fd51003b260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 msgr2=0x7fd52c19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000c00 tx=0x7fd51c011b00 comp rx=0 tx=0).stop 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 shutdown_connections 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd510038da0 0x7fd51003b260 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 --2- 172.21.15.107:0/3723497906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c109800 0x7fd52c19d5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 >> 172.21.15.107:0/3723497906 conn(0x7fd52c07cdf0 msgr2=0x7fd52c106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 shutdown_connections 2024-03-11T19:42:04.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:04.195+0000 7fd531c00700 1 -- 172.21.15.107:0/3723497906 wait complete. 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.507+0000 7f44ef400700 1 Processor -- start 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.507+0000 7f44ef400700 1 -- start start 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.508+0000 7f44ef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.508+0000 7f44ef400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e807b840 con 0x7f44e807ce50 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.508+0000 7f44eca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.508+0000 7f44eca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33924/0 (socket says 172.21.15.107:33924) 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.508+0000 7f44eca00700 1 -- 172.21.15.107:0/2782883730 learned_addr learned my addr 172.21.15.107:0/2782883730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.509+0000 7f44eca00700 1 -- 172.21.15.107:0/2782883730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e807b980 con 0x7f44e807ce50 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.509+0000 7f44eca00700 1 --2- 172.21.15.107:0/2782883730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0098a0 tx=0x7f44dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96781d0675951d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.509+0000 7f44e7400700 1 -- 172.21.15.107:0/2782883730 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44dc004160 con 0x7f44e807ce50 2024-03-11T19:42:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.509+0000 7f44e7400700 1 -- 172.21.15.107:0/2782883730 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44dc0042c0 con 0x7f44e807ce50 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.509+0000 7f44e7400700 1 -- 172.21.15.107:0/2782883730 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44dc011a80 con 0x7f44e807ce50 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 -- 172.21.15.107:0/2782883730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 msgr2=0x7f44e807b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 --2- 172.21.15.107:0/2782883730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0098a0 tx=0x7f44dc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 -- 172.21.15.107:0/2782883730 shutdown_connections 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 --2- 172.21.15.107:0/2782883730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e807b300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 -- 172.21.15.107:0/2782883730 >> 172.21.15.107:0/2782883730 conn(0x7f44e8102be0 msgr2=0x7f44e8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 -- 172.21.15.107:0/2782883730 shutdown_connections 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.510+0000 7f44ef400700 1 -- 172.21.15.107:0/2782883730 wait complete. 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.511+0000 7f44ef400700 1 Processor -- start 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44ef400700 1 -- start start 2024-03-11T19:42:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44ef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44ef400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e819da90 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44eca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44eca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33940/0 (socket says 172.21.15.107:33940) 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.512+0000 7f44eca00700 1 -- 172.21.15.107:0/4089525971 learned_addr learned my addr 172.21.15.107:0/4089525971 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44eca00700 1 -- 172.21.15.107:0/4089525971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc009550 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44eca00700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44dc011740 tx=0x7f44dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44dc003ba0 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44dc01a910 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e819dc90 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44dc024940 con 0x7f44e807ce50 2024-03-11T19:42:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.513+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e819e130 con 0x7f44e807ce50 2024-03-11T19:42:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.514+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e810d9a0 con 0x7f44e807ce50 2024-03-11T19:42:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.515+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f44dc01aa80 con 0x7f44e807ce50 2024-03-11T19:42:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.515+0000 7f44e5600700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 0x7f44d003b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.515+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f44dc04d850 con 0x7f44e807ce50 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.516+0000 7f44e7e00700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 0x7f44d003b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.516+0000 7f44e7e00700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 0x7f44d003b260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44d4006fd0 tx=0x7f44d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.520+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f44dc01adf0 con 0x7f44e807ce50 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.848+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f44e81a0a10 con 0x7f44e807ce50 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.853+0000 7f44e5600700 1 -- 172.21.15.107:0/4089525971 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f44dc0186d0 con 0x7f44e807ce50 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.855+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 msgr2=0x7f44d003b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.855+0000 7f44ef400700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 0x7f44d003b260 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44d4006fd0 tx=0x7f44d4006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.855+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 msgr2=0x7f44e819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.855+0000 7f44ef400700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44dc011740 tx=0x7f44dc011b00 comp rx=0 tx=0).stop 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 shutdown_connections 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f44d0038da0 0x7f44d003b260 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 --2- 172.21.15.107:0/4089525971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e807ce50 0x7f44e819d550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 >> 172.21.15.107:0/4089525971 conn(0x7f44e8102be0 msgr2=0x7f44e81036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 shutdown_connections 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-11T19:42:05.856+0000 7f44ef400700 1 -- 172.21.15.107:0/4089525971 wait complete. 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-03-11T19:42:06.621 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.621 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-11T19:42:06.621 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-03-11T19:42:06.711 INFO:tasks.cephadm:Fetching config... 2024-03-11T19:42:06.712 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:42:06.712 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-11T19:42:06.757 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-11T19:42:06.757 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:42:06.757 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-11T19:42:06.837 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-11T19:42:06.838 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:42:06.838 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-11T19:42:06.972 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-11T19:42:06.973 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:42:06.973 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-11T19:42:07.017 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-11T19:42:07.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:42:07.109 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:06 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4089525971' entity='client.admin' 2024-03-11T19:42:07.189 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:42:07.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:42:07.288 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:42:07.305 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:42:07.354 INFO:teuthology.orchestra.run.smithi145.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGbhczCaUEgrS0RmM7rneMVCBfqLIkpl0XrTDT1DcGfuqqkPAHVCcLSSpXnRyliijWPrH6yNWEqp7Ho4TzrvUOZVfkaOJyfc5wDfAkfmxCcY8sIEZ0m9CSCrwnu7mqc4H7jFDpyVkJPy3VlsEdU2IxKMxz5Wqmuyk+dxvKkI8EAJrN1chBIPnyo/OGufdsIbe2lxoq1/UumIBFl9w7EhQkxGX3bAdohTWPB7eFT2M7LP+NqvtmtWxf3GKL5TWv68XduCa6+h4od2H6OmWkVrZwIKZik2p6wSBEzHMxzOdC0tpT6LJaXXizbJGxDEgqcxKzB1cTqT5Ulp5RCkuM7Zistlik1ngC63hOHGHwl5IBnwX6eqkRUxsMmkHUazT52O4PrcSdicOp2/2EkR139+Sb2p2vSHhOkrNG1v5nell9XvlizuMPOp8pqPar7LSxcjfXT6XxYJ6diQjfNeQRPc1j3uwtprgcmcAJau9+5fOYls3rymn1J270tQA17+ffrVM= ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:42:07.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-11T19:42:07.788 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 -- 172.21.15.107:0/3779915178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 msgr2=0x7f30b8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3779915178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8103ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009910 tx=0x7f30ac009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 -- 172.21.15.107:0/3779915178 shutdown_connections 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3779915178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8103ce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 -- 172.21.15.107:0/3779915178 >> 172.21.15.107:0/3779915178 conn(0x7f30b80fee60 msgr2=0x7f30b81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 -- 172.21.15.107:0/3779915178 shutdown_connections 2024-03-11T19:42:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.699+0000 7f30bfc00700 1 -- 172.21.15.107:0/3779915178 wait complete. 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.700+0000 7f30bfc00700 1 Processor -- start 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bfc00700 1 -- start start 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bfc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bfc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81992e0 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bd200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bd200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33960/0 (socket says 172.21.15.107:33960) 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bd200700 1 -- 172.21.15.107:0/3550540202 learned_addr learned my addr 172.21.15.107:0/3550540202 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.701+0000 7f30bd200700 1 -- 172.21.15.107:0/3550540202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0095f0 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30bd200700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30ac004f40 tx=0x7f30ac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ac01d070 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30ac0044a0 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ac003a40 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b81994e0 con 0x7f30b81038c0 2024-03-11T19:42:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.702+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b8199980 con 0x7f30b81038c0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.703+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f30ac003d00 con 0x7f30b81038c0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.704+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8192720 con 0x7f30b81038c0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.704+0000 7f30b5600700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 0x7f30a003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.704+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30ac04cc40 con 0x7f30b81038c0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.704+0000 7f30b7e00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 0x7f30a003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.705+0000 7f30b7e00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 0x7f30a003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.710+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30ac029360 con 0x7f30b81038c0 2024-03-11T19:42:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.931+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f30b80632a0 con 0x7f30b81038c0 2024-03-11T19:42:10.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.936+0000 7f30b5600700 1 -- 172.21.15.107:0/3550540202 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f30ac026070 con 0x7f30b81038c0 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.938+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 msgr2=0x7f30a003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.938+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 0x7f30a003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 msgr2=0x7f30b8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30ac004f40 tx=0x7f30ac005020 comp rx=0 tx=0).stop 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 shutdown_connections 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30a0038d50 0x7f30a003b210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 --2- 172.21.15.107:0/3550540202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81038c0 0x7f30b8198da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 >> 172.21.15.107:0/3550540202 conn(0x7f30b80fee60 msgr2=0x7f30b80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 shutdown_connections 2024-03-11T19:42:10.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:10.939+0000 7f30bfc00700 1 -- 172.21.15.107:0/3550540202 wait complete. 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:11.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3550540202' entity='client.admin' 2024-03-11T19:42:11.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:11.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:11.841 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-11T19:42:11.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-11T19:42:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:15.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 -- 172.21.15.107:0/2164709383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 msgr2=0x7f3fe81018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/2164709383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe81018e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009910 tx=0x7f3fd8009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 -- 172.21.15.107:0/2164709383 shutdown_connections 2024-03-11T19:42:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/2164709383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe81018e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 -- 172.21.15.107:0/2164709383 >> 172.21.15.107:0/2164709383 conn(0x7f3fe80fca80 msgr2=0x7f3fe80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 -- 172.21.15.107:0/2164709383 shutdown_connections 2024-03-11T19:42:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.115+0000 7f3fe7e00700 1 -- 172.21.15.107:0/2164709383 wait complete. 2024-03-11T19:42:15.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.116+0000 7f3fe7e00700 1 Processor -- start 2024-03-11T19:42:15.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe7e00700 1 -- start start 2024-03-11T19:42:15.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe7e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81991c0 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37380/0 (socket says 172.21.15.107:37380) 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.117+0000 7f3fe6a00700 1 -- 172.21.15.107:0/524573054 learned_addr learned my addr 172.21.15.107:0/524573054 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fe6a00700 1 -- 172.21.15.107:0/524573054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80095f0 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fe6a00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8004f40 tx=0x7f3fd8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd801d070 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fd80044a0 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe81993c0 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8003a40 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.118+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe8199860 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.119+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f3fd8003d00 con 0x7f3fe81014c0 2024-03-11T19:42:15.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.119+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8192720 con 0x7f3fe81014c0 2024-03-11T19:42:15.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.120+0000 7f3fdfe00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 0x7f3fcc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:15.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.120+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fd804cc80 con 0x7f3fe81014c0 2024-03-11T19:42:15.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.120+0000 7f3fde000700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 0x7f3fcc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:15.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.121+0000 7f3fde000700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 0x7f3fcc03b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:15.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.125+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fd8029360 con 0x7f3fe81014c0 2024-03-11T19:42:15.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.349+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fe802d7d0 con 0x7f3fcc038d50 2024-03-11T19:42:15.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.357+0000 7f3fdfe00700 1 -- 172.21.15.107:0/524573054 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3fe802d7d0 con 0x7f3fcc038d50 2024-03-11T19:42:15.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.359+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 msgr2=0x7f3fcc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:15.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.359+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 0x7f3fcc03b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0006fd0 tx=0x7f3fd0006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:15.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.359+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 msgr2=0x7f3fe8198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:15.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.359+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8004f40 tx=0x7f3fd8005020 comp rx=0 tx=0).stop 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 shutdown_connections 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3fcc038d50 0x7f3fcc03b210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/524573054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe81014c0 0x7f3fe8198c80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 >> 172.21.15.107:0/524573054 conn(0x7f3fe80fca80 msgr2=0x7f3fe80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 shutdown_connections 2024-03-11T19:42:15.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:15.360+0000 7f3fe7e00700 1 -- 172.21.15.107:0/524573054 wait complete. 2024-03-11T19:42:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:15 smithi107 ceph-mon[31712]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:16.228 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi122 2024-03-11T19:42:16.228 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:42:16.228 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:42:16.245 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:42:16.246 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:16.303 INFO:tasks.cephadm:Adding host smithi122 to orchestrator... 2024-03-11T19:42:16.304 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch host add smithi122 2024-03-11T19:42:16.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:42:16.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:16.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:16.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:16.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:16.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:16 smithi107 ceph-mon[31712]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:42:16.717 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:17 smithi107 ceph-mon[31712]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:42:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:17 smithi107 ceph-mon[31712]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:18.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:18 smithi107 ceph-mon[31712]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:18.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:18 smithi107 ceph-mon[31712]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:42:18.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:18.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:18.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 -- 172.21.15.107:0/2820442971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 msgr2=0x7f5a601082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 --2- 172.21.15.107:0/2820442971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a601082b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009910 tx=0x7f5a50009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 -- 172.21.15.107:0/2820442971 shutdown_connections 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 --2- 172.21.15.107:0/2820442971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a601082b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 -- 172.21.15.107:0/2820442971 >> 172.21.15.107:0/2820442971 conn(0x7f5a600774a0 msgr2=0x7f5a600798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 -- 172.21.15.107:0/2820442971 shutdown_connections 2024-03-11T19:42:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.769+0000 7f5a66c00700 1 -- 172.21.15.107:0/2820442971 wait complete. 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.770+0000 7f5a66c00700 1 Processor -- start 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a66c00700 1 -- start start 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a66c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a66c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6019db30 con 0x7f5a60107ed0 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a5fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a5fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37404/0 (socket says 172.21.15.107:37404) 2024-03-11T19:42:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.771+0000 7f5a5fe00700 1 -- 172.21.15.107:0/1172385425 learned_addr learned my addr 172.21.15.107:0/1172385425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a5fe00700 1 -- 172.21.15.107:0/1172385425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500095f0 con 0x7f5a60107ed0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a5fe00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004f40 tx=0x7f5a50005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a5001d070 con 0x7f5a60107ed0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5000bc30 con 0x7f5a60107ed0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6019dd30 con 0x7f5a60107ed0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a50003970 con 0x7f5a60107ed0 2024-03-11T19:42:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.772+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6019e150 con 0x7f5a60107ed0 2024-03-11T19:42:18.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.774+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a600632a0 con 0x7f5a60107ed0 2024-03-11T19:42:18.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.774+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f5a50003c30 con 0x7f5a60107ed0 2024-03-11T19:42:18.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.774+0000 7f5a5cc00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 0x7f5a4403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:18.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.774+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a5004cbd0 con 0x7f5a60107ed0 2024-03-11T19:42:18.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.774+0000 7f5a5f400700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 0x7f5a4403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:18.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.775+0000 7f5a5f400700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 0x7f5a4403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c006fd0 tx=0x7f5a4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:18.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:18.780+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a50029540 con 0x7f5a60107ed0 2024-03-11T19:42:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:19.004+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a60112190 con 0x7f5a44038d50 2024-03-11T19:42:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:19 smithi107 ceph-mon[31712]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:20.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:20 smithi107 ceph-mon[31712]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:42:21.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:21 smithi107 ceph-mon[31712]: Deploying cephadm binary to smithi122 2024-03-11T19:42:21.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:21 smithi107 ceph-mon[31712]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:23.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:23 smithi107 ceph-mon[31712]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:23.727 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi122' with addr '172.21.15.122' 2024-03-11T19:42:23.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.721+0000 7f5a5cc00700 1 -- 172.21.15.107:0/1172385425 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5a60112190 con 0x7f5a44038d50 2024-03-11T19:42:23.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.725+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 msgr2=0x7f5a4403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:23.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.725+0000 7f5a66c00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 0x7f5a4403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c006fd0 tx=0x7f5a4c006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.725+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 msgr2=0x7f5a6019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.725+0000 7f5a66c00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004f40 tx=0x7f5a50005020 comp rx=0 tx=0).stop 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 shutdown_connections 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5a44038d50 0x7f5a4403b210 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 --2- 172.21.15.107:0/1172385425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a60107ed0 0x7f5a6019d5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 >> 172.21.15.107:0/1172385425 conn(0x7f5a600774a0 msgr2=0x7f5a60109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 shutdown_connections 2024-03-11T19:42:23.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:23.726+0000 7f5a66c00700 1 -- 172.21.15.107:0/1172385425 wait complete. 2024-03-11T19:42:24.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:42:24.829 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:24.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:24.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:24 smithi107 ceph-mon[31712]: Added host smithi122 2024-03-11T19:42:24.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:24.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:24 smithi107 ceph-mon[31712]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:24.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:25.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:26.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.890+0000 7f69c4600700 1 -- 172.21.15.107:0/1020659203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 msgr2=0x7f69bc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:26.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.890+0000 7f69c4600700 1 --2- 172.21.15.107:0/1020659203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc1082b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009910 tx=0x7f69b0009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.890+0000 7f69c4600700 1 -- 172.21.15.107:0/1020659203 shutdown_connections 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.890+0000 7f69c4600700 1 --2- 172.21.15.107:0/1020659203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc1082b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.890+0000 7f69c4600700 1 -- 172.21.15.107:0/1020659203 >> 172.21.15.107:0/1020659203 conn(0x7f69bc0774a0 msgr2=0x7f69bc0798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.891+0000 7f69c4600700 1 -- 172.21.15.107:0/1020659203 shutdown_connections 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.891+0000 7f69c4600700 1 -- 172.21.15.107:0/1020659203 wait complete. 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.892+0000 7f69c4600700 1 Processor -- start 2024-03-11T19:42:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.892+0000 7f69c4600700 1 -- start start 2024-03-11T19:42:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.892+0000 7f69c4600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.892+0000 7f69c4600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc19db20 con 0x7f69bc107ed0 2024-03-11T19:42:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69c1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69c1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41044/0 (socket says 172.21.15.107:41044) 2024-03-11T19:42:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69c1c00700 1 -- 172.21.15.107:0/3484254168 learned_addr learned my addr 172.21.15.107:0/3484254168 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69c1c00700 1 -- 172.21.15.107:0/3484254168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00095f0 con 0x7f69bc107ed0 2024-03-11T19:42:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69c1c00700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006010 tx=0x7f69b0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.893+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69b001d070 con 0x7f69bc107ed0 2024-03-11T19:42:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.894+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69bc19dd20 con 0x7f69bc107ed0 2024-03-11T19:42:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.894+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69b000bc30 con 0x7f69bc107ed0 2024-03-11T19:42:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.894+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69b0003930 con 0x7f69bc107ed0 2024-03-11T19:42:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.894+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc19e140 con 0x7f69bc107ed0 2024-03-11T19:42:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.895+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f69b0003b30 con 0x7f69bc107ed0 2024-03-11T19:42:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.895+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc0632a0 con 0x7f69bc107ed0 2024-03-11T19:42:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.895+0000 7f69ba000700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 0x7f69a403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.895+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f69b004cc60 con 0x7f69bc107ed0 2024-03-11T19:42:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.896+0000 7f69c1200700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 0x7f69a403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:26.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.896+0000 7f69c1200700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 0x7f69a403b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:26.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:26.901+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69b0029950 con 0x7f69bc107ed0 2024-03-11T19:42:27.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.125+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69bc112190 con 0x7f69a4038d50 2024-03-11T19:42:27.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.127+0000 7f69ba000700 1 -- 172.21.15.107:0/3484254168 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f69bc112190 con 0x7f69a4038d50 2024-03-11T19:42:27.132 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:27.132 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}] 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.129+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 msgr2=0x7f69a403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 0x7f69a403b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 msgr2=0x7f69bc19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006010 tx=0x7f69b0004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 shutdown_connections 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f69a4038d50 0x7f69a403b210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 --2- 172.21.15.107:0/3484254168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69bc107ed0 0x7f69bc19d5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 >> 172.21.15.107:0/3484254168 conn(0x7f69bc0774a0 msgr2=0x7f69bc109890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 shutdown_connections 2024-03-11T19:42:27.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:27.130+0000 7f69c4600700 1 -- 172.21.15.107:0/3484254168 wait complete. 2024-03-11T19:42:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:27 smithi107 ceph-mon[31712]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:27.969 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi145 2024-03-11T19:42:27.969 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:42:27.969 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:42:27.987 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:42:27.987 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:28.045 INFO:tasks.cephadm:Adding host smithi145 to orchestrator... 2024-03-11T19:42:28.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch host add smithi145 2024-03-11T19:42:28.479 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:28.664 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:28 smithi107 ceph-mon[31712]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:42:29.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:29 smithi107 ceph-mon[31712]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:30.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 -- 172.21.15.107:0/2395828593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 msgr2=0x7f2690102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:30.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 --2- 172.21.15.107:0/2395828593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f2690102bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2684009910 tx=0x7f2684009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:30.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 -- 172.21.15.107:0/2395828593 shutdown_connections 2024-03-11T19:42:30.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 --2- 172.21.15.107:0/2395828593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f2690102bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 -- 172.21.15.107:0/2395828593 >> 172.21.15.107:0/2395828593 conn(0x7f26900fc060 msgr2=0x7f26900fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.525+0000 7f2697400700 1 -- 172.21.15.107:0/2395828593 shutdown_connections 2024-03-11T19:42:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.526+0000 7f2697400700 1 -- 172.21.15.107:0/2395828593 wait complete. 2024-03-11T19:42:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.527+0000 7f2697400700 1 Processor -- start 2024-03-11T19:42:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.527+0000 7f2697400700 1 -- start start 2024-03-11T19:42:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2697400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2697400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690199730 con 0x7f26901027f0 2024-03-11T19:42:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2694a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2694a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41062/0 (socket says 172.21.15.107:41062) 2024-03-11T19:42:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2694a00700 1 -- 172.21.15.107:0/1296561068 learned_addr learned my addr 172.21.15.107:0/1296561068 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:30.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.528+0000 7f2694a00700 1 -- 172.21.15.107:0/1296561068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840095f0 con 0x7f26901027f0 2024-03-11T19:42:30.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f2694a00700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2684006010 tx=0x7f2684004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:30.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268401d070 con 0x7f26901027f0 2024-03-11T19:42:30.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f268400bc30 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26840039a0 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2690199930 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.529+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2690199d50 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.530+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f2684004390 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.530+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26900632a0 con 0x7f26901027f0 2024-03-11T19:42:30.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.531+0000 7f268d600700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 0x7f267803b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:30.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.531+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f268404cab0 con 0x7f26901027f0 2024-03-11T19:42:30.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.531+0000 7f268fe00700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 0x7f267803b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:30.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.532+0000 7f268fe00700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 0x7f267803b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2680006fd0 tx=0x7f2680006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:30.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.536+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f268402ae20 con 0x7f26901027f0 2024-03-11T19:42:30.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:30.761+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}) v1 -- 0x7f269010bbb0 con 0x7f2678038db0 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:30.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:30 smithi107 ceph-mon[31712]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:42:32.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:32.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:32.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:32.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:31 smithi107 ceph-mon[31712]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi145", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:42:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:32 smithi107 ceph-mon[31712]: Deploying cephadm binary to smithi145 2024-03-11T19:42:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:32 smithi107 ceph-mon[31712]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:35.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.494+0000 7f268d600700 1 -- 172.21.15.107:0/1296561068 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f269010bbb0 con 0x7f2678038db0 2024-03-11T19:42:35.500 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi145' with addr '172.21.15.145' 2024-03-11T19:42:35.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:35 smithi107 ceph-mon[31712]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:35.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 msgr2=0x7f267803b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:35.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 0x7f267803b270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2680006fd0 tx=0x7f2680006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:35.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 msgr2=0x7f26901991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2684006010 tx=0x7f2684004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 shutdown_connections 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2678038db0 0x7f267803b270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 --2- 172.21.15.107:0/1296561068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26901027f0 0x7f26901991f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.498+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 >> 172.21.15.107:0/1296561068 conn(0x7f26900fc060 msgr2=0x7f26901079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.499+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 shutdown_connections 2024-03-11T19:42:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:35.499+0000 7f2697400700 1 -- 172.21.15.107:0/1296561068 wait complete. 2024-03-11T19:42:36.195 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:42:36.622 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:36.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:36.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:36 smithi107 ceph-mon[31712]: Added host smithi145 2024-03-11T19:42:36.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:36.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:36.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:36 smithi107 ceph-mon[31712]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:37.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:38.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 -- 172.21.15.107:0/4263536473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 msgr2=0x7fa3c8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:38.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 --2- 172.21.15.107:0/4263536473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8109c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009910 tx=0x7fa3b8009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:38.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 -- 172.21.15.107:0/4263536473 shutdown_connections 2024-03-11T19:42:38.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 --2- 172.21.15.107:0/4263536473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8109c10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:38.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 -- 172.21.15.107:0/4263536473 >> 172.21.15.107:0/4263536473 conn(0x7fa3c80ff360 msgr2=0x7fa3c8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:38.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 -- 172.21.15.107:0/4263536473 shutdown_connections 2024-03-11T19:42:38.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.730+0000 7fa3cd400700 1 -- 172.21.15.107:0/4263536473 wait complete. 2024-03-11T19:42:38.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.732+0000 7fa3cd400700 1 Processor -- start 2024-03-11T19:42:38.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.732+0000 7fa3cd400700 1 -- start start 2024-03-11T19:42:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.732+0000 7fa3cd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.732+0000 7fa3cd400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8199740 con 0x7fa3c8109830 2024-03-11T19:42:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.733+0000 7fa3c6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.733+0000 7fa3c6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38680/0 (socket says 172.21.15.107:38680) 2024-03-11T19:42:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.733+0000 7fa3c6800700 1 -- 172.21.15.107:0/419112918 learned_addr learned my addr 172.21.15.107:0/419112918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.733+0000 7fa3c6800700 1 -- 172.21.15.107:0/419112918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80095f0 con 0x7fa3c8109830 2024-03-11T19:42:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3c6800700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006010 tx=0x7fa3b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3b801d070 con 0x7fa3c8109830 2024-03-11T19:42:38.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3b800bbd0 con 0x7fa3c8109830 2024-03-11T19:42:38.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3b8003880 con 0x7fa3c8109830 2024-03-11T19:42:38.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c81999a0 con 0x7fa3c8109830 2024-03-11T19:42:38.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.734+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c8199e20 con 0x7fa3c8109830 2024-03-11T19:42:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.735+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c8104f70 con 0x7fa3c8109830 2024-03-11T19:42:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.735+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fa3b800bd40 con 0x7fa3c8109830 2024-03-11T19:42:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.736+0000 7fa3bf400700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 0x7fa3ac03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.736+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa3b804cb20 con 0x7fa3c8109830 2024-03-11T19:42:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.736+0000 7fa3c5e00700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 0x7fa3ac03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.737+0000 7fa3c5e00700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 0x7fa3ac03b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.742+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3b802ae20 con 0x7fa3c8109830 2024-03-11T19:42:38.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.969+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3c810dd30 con 0x7fa3ac038d50 2024-03-11T19:42:38.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.972+0000 7fa3bf400700 1 -- 172.21.15.107:0/419112918 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fa3c810dd30 con 0x7fa3ac038d50 2024-03-11T19:42:38.976 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:42:38.978 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.145", "hostname": "smithi145", "labels": [], "status": ""}] 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 msgr2=0x7fa3ac03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 0x7fa3ac03b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 msgr2=0x7fa3c8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006010 tx=0x7fa3b8004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 shutdown_connections 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa3ac038d50 0x7fa3ac03b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 --2- 172.21.15.107:0/419112918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3c8109830 0x7fa3c8199200 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 >> 172.21.15.107:0/419112918 conn(0x7fa3c80ff360 msgr2=0x7fa3c810b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 shutdown_connections 2024-03-11T19:42:38.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:38.975+0000 7fa3cd400700 1 -- 172.21.15.107:0/419112918 wait complete. 2024-03-11T19:42:39.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:39 smithi107 ceph-mon[31712]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:39.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:39.783 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-11T19:42:39.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-11T19:42:40.218 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:42:40.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:40 smithi107 ceph-mon[31712]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:42:41.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:41 smithi107 ceph-mon[31712]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 -- 172.21.15.107:0/2292543544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 msgr2=0x7f7554107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 --2- 172.21.15.107:0/2292543544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554107a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7544009910 tx=0x7f7544009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 -- 172.21.15.107:0/2292543544 shutdown_connections 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 --2- 172.21.15.107:0/2292543544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554107a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 -- 172.21.15.107:0/2292543544 >> 172.21.15.107:0/2292543544 conn(0x7f75540fd140 msgr2=0x7f75540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 -- 172.21.15.107:0/2292543544 shutdown_connections 2024-03-11T19:42:42.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.425+0000 7f755ac00700 1 -- 172.21.15.107:0/2292543544 wait complete. 2024-03-11T19:42:42.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.426+0000 7f755ac00700 1 Processor -- start 2024-03-11T19:42:42.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.426+0000 7f755ac00700 1 -- start start 2024-03-11T19:42:42.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.426+0000 7f755ac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:42.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.426+0000 7f755ac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541997b0 con 0x7f7554107630 2024-03-11T19:42:42.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.427+0000 7f7553e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:42.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.427+0000 7f7553e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38694/0 (socket says 172.21.15.107:38694) 2024-03-11T19:42:42.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.427+0000 7f7553e00700 1 -- 172.21.15.107:0/1356756176 learned_addr learned my addr 172.21.15.107:0/1356756176 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:42:42.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.427+0000 7f7553e00700 1 -- 172.21.15.107:0/1356756176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440095f0 con 0x7f7554107630 2024-03-11T19:42:42.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.427+0000 7f7553e00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7544006010 tx=0x7f7544004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.428+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754401d070 con 0x7f7554107630 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.428+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f754400bbd0 con 0x7f7554107630 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.428+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7544003880 con 0x7f7554107630 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.428+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7554199a10 con 0x7f7554107630 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.428+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7554199e90 con 0x7f7554107630 2024-03-11T19:42:42.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.429+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f754400bd40 con 0x7f7554107630 2024-03-11T19:42:42.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.429+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554102d70 con 0x7f7554107630 2024-03-11T19:42:42.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.430+0000 7f7550c00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 0x7f753803b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:42.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.430+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f754404ca60 con 0x7f7554107630 2024-03-11T19:42:42.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.430+0000 7f7553400700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 0x7f753803b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:42.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.431+0000 7f7553400700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 0x7f753803b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7540006fd0 tx=0x7f7540006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:42.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.436+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f754402ae20 con 0x7f7554107630 2024-03-11T19:42:42.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:42.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:42.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:42.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:42.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi145", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:42:42.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:42.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:42.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:42 smithi107 ceph-mon[31712]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:42:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:42.681+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f75540632a0 con 0x7f7554107630 2024-03-11T19:42:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.544+0000 7f7550c00700 1 -- 172.21.15.107:0/1356756176 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f754402ac40 con 0x7f7554107630 2024-03-11T19:42:43.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 msgr2=0x7f753803b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:43.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 0x7f753803b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7540006fd0 tx=0x7f7540006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:43.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 msgr2=0x7f7554199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:43.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7544006010 tx=0x7f7544004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 shutdown_connections 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7538038db0 0x7f753803b270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 --2- 172.21.15.107:0/1356756176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7554107630 0x7f7554199270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.548+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 >> 172.21.15.107:0/1356756176 conn(0x7f75540fd140 msgr2=0x7f75541090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.549+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 shutdown_connections 2024-03-11T19:42:43.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:42:43.549+0000 7f755ac00700 1 -- 172.21.15.107:0/1356756176 wait complete. 2024-03-11T19:42:43.554 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-03-11T19:42:43.629 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:43.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:43 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1356756176' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-11T19:42:44.389 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-03-11T19:42:44.389 INFO:tasks.cephadm:Adding mon.b on smithi122 2024-03-11T19:42:44.389 INFO:tasks.cephadm:Adding mon.c on smithi145 2024-03-11T19:42:44.389 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi122:172.21.15.122=b;smithi145:172.21.15.145=c' 2024-03-11T19:42:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:42:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:44 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1356756176' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-11T19:42:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:44 smithi107 ceph-mon[31712]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:42:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:44 smithi107 ceph-mon[31712]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 -- 172.21.15.145:0/2594822560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 msgr2=0x7f0254102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 --2- 172.21.15.145:0/2594822560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254102bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0244009910 tx=0x7f0244009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 -- 172.21.15.145:0/2594822560 shutdown_connections 2024-03-11T19:42:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 --2- 172.21.15.145:0/2594822560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254102bd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:45.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 -- 172.21.15.145:0/2594822560 >> 172.21.15.145:0/2594822560 conn(0x7f02540fc020 msgr2=0x7f02540fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:45.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 -- 172.21.15.145:0/2594822560 shutdown_connections 2024-03-11T19:42:45.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 -- 172.21.15.145:0/2594822560 wait complete. 2024-03-11T19:42:45.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.919+0000 7f0259f80700 1 Processor -- start 2024-03-11T19:42:45.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f0259f80700 1 -- start start 2024-03-11T19:42:45.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f0259f80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f0259f80700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254197550 con 0x7f02541027f0 2024-03-11T19:42:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f02537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f02537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.145:53044/0 (socket says 172.21.15.145:53044) 2024-03-11T19:42:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.920+0000 7f02537fe700 1 -- 172.21.15.145:0/951116993 learned_addr learned my addr 172.21.15.145:0/951116993 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:42:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.921+0000 7f02537fe700 1 -- 172.21.15.145:0/951116993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440095f0 con 0x7f02541027f0 2024-03-11T19:42:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.921+0000 7f02537fe700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0244005850 tx=0x7f0244004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.921+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f024401d070 con 0x7f02541027f0 2024-03-11T19:42:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.921+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254197750 con 0x7f02541027f0 2024-03-11T19:42:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.921+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254197b70 con 0x7f02541027f0 2024-03-11T19:42:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.922+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f024400bc30 con 0x7f02541027f0 2024-03-11T19:42:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.922+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0244003920 con 0x7f02541027f0 2024-03-11T19:42:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.922+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0240005330 con 0x7f02541027f0 2024-03-11T19:42:45.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.923+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f024400b750 con 0x7f02541027f0 2024-03-11T19:42:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.923+0000 7f0250ff9700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 0x7f023c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.923+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f024404d070 con 0x7f02541027f0 2024-03-11T19:42:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.924+0000 7f0252ffd700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 0x7f023c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.925+0000 7f0252ffd700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 0x7f023c03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:45.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:45.938+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f024400bda0 con 0x7f02541027f0 2024-03-11T19:42:46.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.152+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi122:172.21.15.122=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0240000c80 con 0x7f023c038d50 2024-03-11T19:42:46.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.167+0000 7f0250ff9700 1 -- 172.21.15.145:0/951116993 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0240000c80 con 0x7f023c038d50 2024-03-11T19:42:46.185 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mon update... 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 msgr2=0x7f023c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 0x7f023c03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 msgr2=0x7f0254197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0244005850 tx=0x7f0244004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 shutdown_connections 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f023c038d50 0x7f023c03b210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 --2- 172.21.15.145:0/951116993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02541027f0 0x7f0254197010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.169+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 >> 172.21.15.145:0/951116993 conn(0x7f02540fc020 msgr2=0x7f02541035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.170+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 shutdown_connections 2024-03-11T19:42:46.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:46.170+0000 7f0259f80700 1 -- 172.21.15.145:0/951116993 wait complete. 2024-03-11T19:42:46.719 DEBUG:teuthology.orchestra.run.smithi122:mon.b> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.b.service 2024-03-11T19:42:46.728 DEBUG:teuthology.orchestra.run.smithi145:mon.c> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.c.service 2024-03-11T19:42:46.730 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:42:46.731 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:42:47.482 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:42:47.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi122:172.21.15.122=b;smithi145:172.21.15.145=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi122:172.21.15.122=b;smithi145:172.21.15.145=c;count:3 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:47.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:47 smithi107 ceph-mon[31712]: Deploying daemon mon.c on smithi145 2024-03-11T19:42:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:49 smithi107 ceph-mon[31712]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:50.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.436+0000 7f7228d80700 1 -- 172.21.15.145:0/838183839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 msgr2=0x7f7224102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:50.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.436+0000 7f7228d80700 1 --2- 172.21.15.145:0/838183839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224102bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7218009910 tx=0x7f7218009c20 comp rx=0 tx=0).stop 2024-03-11T19:42:50.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.436+0000 7f7228d80700 1 -- 172.21.15.145:0/838183839 shutdown_connections 2024-03-11T19:42:50.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.436+0000 7f7228d80700 1 --2- 172.21.15.145:0/838183839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224102bd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.436+0000 7f7228d80700 1 -- 172.21.15.145:0/838183839 >> 172.21.15.145:0/838183839 conn(0x7f72240fc060 msgr2=0x7f72240fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.437+0000 7f7228d80700 1 -- 172.21.15.145:0/838183839 shutdown_connections 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.437+0000 7f7228d80700 1 -- 172.21.15.145:0/838183839 wait complete. 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.437+0000 7f7228d80700 1 Processor -- start 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.437+0000 7f7228d80700 1 -- start start 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.438+0000 7f7228d80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:50.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.438+0000 7f7228d80700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224102340 con 0x7f72241027f0 2024-03-11T19:42:50.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.438+0000 7f722259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:50.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.438+0000 7f722259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.145:38912/0 (socket says 172.21.15.145:38912) 2024-03-11T19:42:50.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.438+0000 7f722259c700 1 -- 172.21.15.145:0/3037843931 learned_addr learned my addr 172.21.15.145:0/3037843931 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:42:50.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.439+0000 7f722259c700 1 -- 172.21.15.145:0/3037843931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180095f0 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.439+0000 7f722259c700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7218005850 tx=0x7f7218004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.439+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f721801d070 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.440+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f721800bc30 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.440+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7224100990 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.440+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7218003a20 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.440+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7224100db0 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.441+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72240632a0 con 0x7f72241027f0 2024-03-11T19:42:50.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.441+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f7218004390 con 0x7f72241027f0 2024-03-11T19:42:50.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.441+0000 7f72137fe700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 0x7f720c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:50.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.441+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f721804cb10 con 0x7f72241027f0 2024-03-11T19:42:50.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.442+0000 7f7221d9b700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 0x7f720c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:50.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.443+0000 7f7221d9b700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 0x7f720c03b270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7214006fd0 tx=0x7f7214006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:50.461 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.457+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7218005c00 con 0x7f72241027f0 2024-03-11T19:42:50.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.765+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7224101060 con 0x7f72241027f0 2024-03-11T19:42:50.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.766+0000 7f72137fe700 1 -- 172.21.15.145:0/3037843931 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7f7218026070 con 0x7f72241027f0 2024-03-11T19:42:50.771 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:42:50.771 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":1,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","modified":"2024-03-11T19:40:40.021426Z","created":"2024-03-11T19:40:40.021426Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 msgr2=0x7f720c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 0x7f720c03b270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7214006fd0 tx=0x7f7214006e40 comp rx=0 tx=0).stop 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 msgr2=0x7f7224100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7218005850 tx=0x7f7218004ff0 comp rx=0 tx=0).stop 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 shutdown_connections 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f720c038db0 0x7f720c03b270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 --2- 172.21.15.145:0/3037843931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72241027f0 0x7f7224100450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 >> 172.21.15.145:0/3037843931 conn(0x7f72240fc060 msgr2=0x7f72241035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:50.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 shutdown_connections 2024-03-11T19:42:50.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:50.768+0000 7f7228d80700 1 -- 172.21.15.145:0/3037843931 wait complete. 2024-03-11T19:42:50.774 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 1 2024-03-11T19:42:51.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:50 smithi145 ceph-mon[36730]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-11T19:42:51.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:51 smithi145 bash[36697]: e60cd6454fc49e50387f439c6c6872f66d1168dbe0dcd58c03ca34f2e99ffdda 2024-03-11T19:42:51.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:51 smithi145 systemd[1]: Started Ceph mon.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:42:52.556 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:42:52.556 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:42:52.797 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: mon.a calling monitor election 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: mon.c calling monitor election 2024-03-11T19:42:56.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: fsmap 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: mgrmap e13: x(active, since 61s) 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: overall HEALTH_OK 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.290 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:42:56.291 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:56.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: mon.a calling monitor election 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: mon.c calling monitor election 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:56.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: fsmap 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: mgrmap e13: x(active, since 61s) 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: overall HEALTH_OK 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:42:56.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:56 smithi145 ceph-mon[36730]: Deploying daemon mon.b on smithi122 2024-03-11T19:42:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:56 smithi145 ceph-mon[36730]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:56 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:57.039 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 -- 172.21.15.145:0/765864231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33b00035c0 msgr2=0x7f33b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:57.039 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 --2- 172.21.15.145:0/765864231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33b00035c0 0x7f33b0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33c4101680 tx=0x7f33c000feb0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 -- 172.21.15.145:0/765864231 shutdown_connections 2024-03-11T19:42:57.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 --2- 172.21.15.145:0/765864231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33b00035c0 0x7f33b0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 -- 172.21.15.145:0/765864231 >> 172.21.15.145:0/765864231 conn(0x7f33c40fd240 msgr2=0x7f33c40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:57.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 -- 172.21.15.145:0/765864231 shutdown_connections 2024-03-11T19:42:57.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.035+0000 7f33cc39f700 1 -- 172.21.15.145:0/765864231 wait complete. 2024-03-11T19:42:57.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 Processor -- start 2024-03-11T19:42:57.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 -- start start 2024-03-11T19:42:57.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:57.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 0x7f33c419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4197bd0 con 0x7f33c4197640 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33cc39f700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4197d40 con 0x7f33b00035c0 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33ca13b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33ca13b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:34702/0 (socket says 172.21.15.145:34702) 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.036+0000 7f33ca13b700 1 -- 172.21.15.145:0/3770101653 learned_addr learned my addr 172.21.15.145:0/3770101653 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33ca13b700 1 -- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 msgr2=0x7f33c4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33ca13b700 1 -- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 msgr2=0x7f33c4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33ca13b700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33ca13b700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33c993a700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 0x7f33c419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33c993a700 1 -- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 msgr2=0x7f33c4197100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:42:57.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33c993a700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.037+0000 7f33c993a700 1 -- 172.21.15.145:0/3770101653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c000f900 con 0x7f33c4197640 2024-03-11T19:42:57.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33c993a700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 0x7f33c419b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b4d0 tx=0x7f33b400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:57.045 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f33b4011630 con 0x7f33c4197640 2024-03-11T19:42:57.045 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b4011c70 con 0x7f33c4197640 2024-03-11T19:42:57.045 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c419bf70 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f33b4010de0 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.038+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c419c4c0 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.039+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4190ff0 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.040+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f33b40104a0 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.040+0000 7f33bb7fe700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 0x7f33a803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.040+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f33b404b4b0 con 0x7f33c4197640 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.041+0000 7f33ca13b700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 0x7f33a803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.042+0000 7f33ca13b700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 0x7f33a803b000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33c000f800 tx=0x7f33c0005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:57.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.058+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33b4011de0 con 0x7f33c4197640 2024-03-11T19:42:57.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:56 smithi107 ceph-mon[31712]: Deploying daemon mon.b on smithi122 2024-03-11T19:42:57.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:56 smithi107 ceph-mon[31712]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:57.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:56 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:42:57.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.332+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f33c4198460 con 0x7f33c4197640 2024-03-11T19:42:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.333+0000 7f33bb7fe700 1 -- 172.21.15.145:0/3770101653 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f33b400d1c0 con 0x7f33c4197640 2024-03-11T19:42:57.337 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:42:57.337 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","modified":"2024-03-11T19:42:50.864605Z","created":"2024-03-11T19:40:40.021426Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:42:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 msgr2=0x7f33a803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 0x7f33a803b000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33c000f800 tx=0x7f33c0005c20 comp rx=0 tx=0).stop 2024-03-11T19:42:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 msgr2=0x7f33c419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 0x7f33c419b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b4d0 tx=0x7f33b400b890 comp rx=0 tx=0).stop 2024-03-11T19:42:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 shutdown_connections 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33a8038b40 0x7f33a803b000 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33b00035c0 0x7f33c4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 --2- 172.21.15.145:0/3770101653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c4197640 0x7f33c419b9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 >> 172.21.15.145:0/3770101653 conn(0x7f33c40fd240 msgr2=0x7f33c40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 shutdown_connections 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:42:57.335+0000 7f33cc39f700 1 -- 172.21.15.145:0/3770101653 wait complete. 2024-03-11T19:42:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 2 2024-03-11T19:42:58.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:57 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/3770101653' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:42:58.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:57 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/3770101653' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:42:58.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:42:57 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:42:57.865+0000 7f0a7c600700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-11T19:42:58.851 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:42:58.851 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:42:59.084 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:42:59.205 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:42:58 smithi145 ceph-mon[36730]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:42:59.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:42:58 smithi107 ceph-mon[31712]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:00.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:42:59 smithi122 systemd[1]: Started Ceph mon.b for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:43:04.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.891+0000 7f80f9ed1700 1 -- 172.21.15.145:0/2102549323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0035c0 msgr2=0x7f80dc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.891+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/2102549323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0035c0 0x7f80dc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80f4069f30 tx=0x7f80e400fe60 comp rx=0 tx=0).stop 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.891+0000 7f80f9ed1700 1 -- 172.21.15.145:0/2102549323 shutdown_connections 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.891+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/2102549323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0035c0 0x7f80dc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.891+0000 7f80f9ed1700 1 -- 172.21.15.145:0/2102549323 >> 172.21.15.145:0/2102549323 conn(0x7f80f40fc180 msgr2=0x7f80f40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.892+0000 7f80f9ed1700 1 -- 172.21.15.145:0/2102549323 shutdown_connections 2024-03-11T19:43:04.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.892+0000 7f80f9ed1700 1 -- 172.21.15.145:0/2102549323 wait complete. 2024-03-11T19:43:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.892+0000 7f80f9ed1700 1 Processor -- start 2024-03-11T19:43:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 -- start start 2024-03-11T19:43:04.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 0x7f80f419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4197f20 con 0x7f80f419c0b0 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a3e20 con 0x7f80f4197830 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f9ed1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a3f60 con 0x7f80dc0035c0 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.893+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59432/0 (socket says 172.21.15.145:59432) 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f37fe700 1 -- 172.21.15.145:0/697299632 learned_addr learned my addr 172.21.15.145:0/697299632 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:43:04.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f37fe700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 msgr2=0x7f80f41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f37fe700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 msgr2=0x7f80f41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f37fe700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f37fe700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f2ffd700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 0x7f80f419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f2ffd700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 msgr2=0x7f80f419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f2ffd700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 msgr2=0x7f80f419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f2ffd700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f2ffd700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 msgr2=0x7f80f41972f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:04.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 msgr2=0x7f80f419bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:43:04.901 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:04.901 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.894+0000 7f80f3fff700 1 -- 172.21.15.145:0/697299632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e400f8b0 con 0x7f80f419c0b0 2024-03-11T19:43:04.901 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.895+0000 7f80f3fff700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 0x7f80f419c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00bd50 tx=0x7f80ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.899+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00cd10 con 0x7f80f419c0b0 2024-03-11T19:43:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.899+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f41a4240 con 0x7f80f419c0b0 2024-03-11T19:43:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.899+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a4790 con 0x7f80f419c0b0 2024-03-11T19:43:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.900+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80ec009870 con 0x7f80f419c0b0 2024-03-11T19:43:04.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.900+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec01d820 con 0x7f80f419c0b0 2024-03-11T19:43:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.901+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc002b30 con 0x7f80f419c0b0 2024-03-11T19:43:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.901+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f80ec01d980 con 0x7f80f419c0b0 2024-03-11T19:43:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.901+0000 7f80f0ff9700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 0x7f80e003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:04.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.901+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f80ec04b400 con 0x7f80f419c0b0 2024-03-11T19:43:04.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.901+0000 7f80f37fe700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 0x7f80e003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:04.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.902+0000 7f80f37fe700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 0x7f80e003b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80e4005850 tx=0x7f80e4005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:04.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:04.917+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80ec01c360 con 0x7f80f419c0b0 2024-03-11T19:43:05.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.189+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f80dc002de0 con 0x7f80f419c0b0 2024-03-11T19:43:05.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: mon.c calling monitor election 2024-03-11T19:43:05.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:43:05.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: mon.a calling monitor election 2024-03-11T19:43:05.193 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:43:05.194 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: fsmap 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: mgrmap e13: x(active, since 70s) 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: mon.b (rank 2) addr [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] is down (out of quorum) 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:05.195 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:05.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.190+0000 7f80f0ff9700 1 -- 172.21.15.145:0/697299632 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1287 (secure 0 0 0) 0x7f80ec019070 con 0x7f80f419c0b0 2024-03-11T19:43:05.196 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-03-11T19:43:05.196 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","modified":"2024-03-11T19:42:59.868261Z","created":"2024-03-11T19:40:40.021426Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:3300","nonce":0},{"type":"v1","addr":"172.21.15.145:6789","nonce":0}]},"addr":"172.21.15.145:6789/0","public_addr":"172.21.15.145:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 msgr2=0x7f80e003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 0x7f80e003b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80e4005850 tx=0x7f80e4005de0 comp rx=0 tx=0).stop 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 msgr2=0x7f80f419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 0x7f80f419c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00bd50 tx=0x7f80ec00be30 comp rx=0 tx=0).stop 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 shutdown_connections 2024-03-11T19:43:05.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80e00391e0 0x7f80e003b6a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80f419c0b0 0x7f80f419c560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80dc0035c0 0x7f80f41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 --2- 172.21.15.145:0/697299632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80f4197830 0x7f80f419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 >> 172.21.15.145:0/697299632 conn(0x7f80f40fc180 msgr2=0x7f80f40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 shutdown_connections 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:05.193+0000 7f80f9ed1700 1 -- 172.21.15.145:0/697299632 wait complete. 2024-03-11T19:43:05.198 INFO:teuthology.orchestra.run.smithi145.stderr:dumped monmap epoch 3 2024-03-11T19:43:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: mon.c calling monitor election 2024-03-11T19:43:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:43:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: mon.a calling monitor election 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: fsmap 2024-03-11T19:43:05.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: mgrmap e13: x(active, since 70s) 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: mon.b (rank 2) addr [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] is down (out of quorum) 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:05.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:05.841 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-11T19:43:05.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-11T19:43:06.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:06.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:06.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:05 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/697299632' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:43:06.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:06.165 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:06.273 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:43:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:05 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/697299632' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:43:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: mon.c calling monitor election 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: mon.a calling monitor election 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:43:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: fsmap 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: mgrmap e13: x(active, since 70s) 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:43:07.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: mon.b (rank 2) addr [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] is down (out of quorum) 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/697299632' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:07.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mon.b calling monitor election 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:07.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mon.b calling monitor election 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mon.c calling monitor election 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mon.a calling monitor election 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: fsmap 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: mgrmap e13: x(active, since 72s) 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Cluster is now healthy 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: overall HEALTH_OK 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:07 smithi107 ceph-mon[31712]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mon.b calling monitor election 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mon.b calling monitor election 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mon.c calling monitor election 2024-03-11T19:43:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mon.a calling monitor election 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: fsmap 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: mgrmap e13: x(active, since 72s) 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Cluster is now healthy 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: overall HEALTH_OK 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:07 smithi122 ceph-mon[36397]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mon.b calling monitor election 2024-03-11T19:43:08.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mon.b calling monitor election 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mon.c calling monitor election 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mon.a calling monitor election 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0],c=[v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: fsmap 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: mgrmap e13: x(active, since 72s) 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:43:08.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Cluster is now healthy 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: overall HEALTH_OK 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:08.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:07 smithi145 ceph-mon[36730]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.527+0000 7f8a33000700 1 -- 172.21.15.107:0/316589543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c1065f0 msgr2=0x7f8a2c1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.527+0000 7f8a33000700 1 --2- 172.21.15.107:0/316589543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c1069d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009910 tx=0x7f8a1c009c20 comp rx=0 tx=0).stop 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.527+0000 7f8a33000700 1 -- 172.21.15.107:0/316589543 shutdown_connections 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.527+0000 7f8a33000700 1 --2- 172.21.15.107:0/316589543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c1069d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.527+0000 7f8a33000700 1 -- 172.21.15.107:0/316589543 >> 172.21.15.107:0/316589543 conn(0x7f8a2c077310 msgr2=0x7f8a2c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.528+0000 7f8a33000700 1 -- 172.21.15.107:0/316589543 shutdown_connections 2024-03-11T19:43:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.528+0000 7f8a33000700 1 -- 172.21.15.107:0/316589543 wait complete. 2024-03-11T19:43:08.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.529+0000 7f8a33000700 1 Processor -- start 2024-03-11T19:43:08.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.529+0000 7f8a33000700 1 -- start start 2024-03-11T19:43:08.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:08.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 0x7f8a2c1afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a2c1958d0 0x7f8a2c1b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1b2890 con 0x7f8a2c195450 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1b2a00 con 0x7f8a2c1958d0 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a33000700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1b2bc0 con 0x7f8a2c1065f0 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a2be00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a2b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 0x7f8a2c1afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:08.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a2be00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45084/0 (socket says 172.21.15.107:45084) 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a2be00700 1 -- 172.21.15.107:0/2380402730 learned_addr learned my addr 172.21.15.107:0/2380402730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.530+0000 7f8a31000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a2c1958d0 0x7f8a2c1b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 -- 172.21.15.107:0/2380402730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a2c1958d0 msgr2=0x7f8a2c1b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a2c1958d0 0x7f8a2c1b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 -- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 msgr2=0x7f8a2c1afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 0x7f8a2c1afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 -- 172.21.15.107:0/2380402730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c0095f0 con 0x7f8a2c1065f0 2024-03-11T19:43:08.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2b400700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 0x7f8a2c1afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a2be00700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00bcf0 tx=0x7f8a1c00bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.531+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c01d070 con 0x7f8a2c1065f0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.532+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a1c00fb50 con 0x7f8a2c1065f0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.532+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c1b2ea0 con 0x7f8a2c1065f0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.532+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c022d70 con 0x7f8a2c1065f0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.532+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c1b3310 con 0x7f8a2c1065f0 2024-03-11T19:43:08.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.533+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f8a1c017440 con 0x7f8a2c1065f0 2024-03-11T19:43:08.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.533+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c101c60 con 0x7f8a2c1065f0 2024-03-11T19:43:08.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.534+0000 7f8a28c00700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 0x7f8a1003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:08.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.534+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a1c04ced0 con 0x7f8a2c1065f0 2024-03-11T19:43:08.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.535+0000 7f8a2b400700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 0x7f8a1003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:08.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.535+0000 7f8a2b400700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 0x7f8a1003b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c196540 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.539+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a1c020920 con 0x7f8a2c1065f0 2024-03-11T19:43:08.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.760+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8a2c10a9a0 con 0x7f8a2c1065f0 2024-03-11T19:43:08.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.761+0000 7f8a28c00700 1 -- 172.21.15.107:0/2380402730 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f8a1c04e070 con 0x7f8a2c1065f0 2024-03-11T19:43:08.766 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:43:08.766 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-03-11T19:43:08.766 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:43:08.766 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] 2024-03-11T19:43:08.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 msgr2=0x7f8a1003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:08.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 0x7f8a1003b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c196540 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2024-03-11T19:43:08.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 msgr2=0x7f8a2c194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:08.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00bcf0 tx=0x7f8a1c00bd20 comp rx=0 tx=0).stop 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 shutdown_connections 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8a100391e0 0x7f8a1003b6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a2c1958d0 0x7f8a2c1b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c1065f0 0x7f8a2c194f10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 --2- 172.21.15.107:0/2380402730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a2c195450 0x7f8a2c1afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.763+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 >> 172.21.15.107:0/2380402730 conn(0x7f8a2c077310 msgr2=0x7f8a2c069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.764+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 shutdown_connections 2024-03-11T19:43:08.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:08.764+0000 7f8a33000700 1 -- 172.21.15.107:0/2380402730 wait complete. 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: Reconfiguring daemon mon.a on smithi107 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:09.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:08 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2380402730' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: Reconfiguring daemon mon.a on smithi107 2024-03-11T19:43:09.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:09.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:08 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2380402730' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:43:09.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: Reconfiguring daemon mon.a on smithi107 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:09.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:08 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2380402730' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:09.636 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-11T19:43:09.636 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:43:09.636 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:43:09.723 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:43:09.723 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:43:09.855 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:43:09.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:43:09.893 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:43:09.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:43:09.965 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:43:09.965 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:43:10.001 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:43:10.002 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:43:10.075 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-03-11T19:43:10.075 INFO:tasks.cephadm:Adding mgr.y on smithi122 2024-03-11T19:43:10.076 INFO:tasks.cephadm:Adding mgr.z on smithi145 2024-03-11T19:43:10.076 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch apply mgr '3;smithi107=x;smithi122=y;smithi145=z' 2024-03-11T19:43:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:43:10.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:10.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:11.286 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:11.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 -- 172.21.15.145:0/1606001240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c1006d0 msgr2=0x7fa58c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 --2- 172.21.15.145:0/1606001240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c1006d0 0x7fa58c100af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b0a0 tx=0x7fa58400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 -- 172.21.15.145:0/1606001240 shutdown_connections 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 --2- 172.21.15.145:0/1606001240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c1006d0 0x7fa58c100af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 -- 172.21.15.145:0/1606001240 >> 172.21.15.145:0/1606001240 conn(0x7fa58c0fbc50 msgr2=0x7fa58c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 -- 172.21.15.145:0/1606001240 shutdown_connections 2024-03-11T19:43:11.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.762+0000 7fa593647700 1 -- 172.21.15.145:0/1606001240 wait complete. 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 Processor -- start 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 -- start start 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c1006d0 0x7fa58c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c196260 0x7fa58c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c19e010 con 0x7fa58c196260 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c19e180 con 0x7fa58c1006d0 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.763+0000 7fa593647700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c19e340 con 0x7fa58c1966e0 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:38658/0 (socket says 172.21.15.145:38658) 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 -- 172.21.15.145:0/3690911802 learned_addr learned my addr 172.21.15.145:0/3690911802 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa5913e3700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c1006d0 0x7fa58c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:11.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 -- 172.21.15.145:0/3690911802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c1006d0 msgr2=0x7fa58c195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c1006d0 0x7fa58c195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 -- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c196260 msgr2=0x7fa58c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c196260 0x7fa58c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 -- 172.21.15.145:0/3690911802 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009d00 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa591be4700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b550 tx=0x7fa58800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588011830 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c19e620 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa588011e70 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800f2e0 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.764+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c19eba0 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.765+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fa58800f440 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.765+0000 7fa5827fc700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 0x7fa57803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.765+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa588010380 con 0x7fa58c1966e0 2024-03-11T19:43:11.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.765+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c18f4e0 con 0x7fa58c1966e0 2024-03-11T19:43:11.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.766+0000 7fa5913e3700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 0x7fa57803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:11.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.767+0000 7fa5913e3700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 0x7fa57803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa58c197200 tx=0x7fa584005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:11.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.771+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5880174a0 con 0x7fa58c1966e0 2024-03-11T19:43:11.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.974+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi122=y;smithi145=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa58c02d7d0 con 0x7fa5780391e0 2024-03-11T19:43:11.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.990+0000 7fa5827fc700 1 -- 172.21.15.145:0/3690911802 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa58c02d7d0 con 0x7fa5780391e0 2024-03-11T19:43:11.995 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mgr update... 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 msgr2=0x7fa57803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 0x7fa57803b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa58c197200 tx=0x7fa584005040 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 msgr2=0x7fa58c19d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b550 tx=0x7fa58800b860 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 shutdown_connections 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa5780391e0 0x7fa57803b6a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa58c1966e0 0x7fa58c19d840 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c1006d0 0x7fa58c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 --2- 172.21.15.145:0/3690911802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa58c196260 0x7fa58c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:11.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 >> 172.21.15.145:0/3690911802 conn(0x7fa58c0fbc50 msgr2=0x7fa58c0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:11.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 shutdown_connections 2024-03-11T19:43:11.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:43:11.992+0000 7fa593647700 1 -- 172.21.15.145:0/3690911802 wait complete. 2024-03-11T19:43:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:12 smithi122 ceph-mon[36397]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:43:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:12 smithi122 ceph-mon[36397]: Reconfiguring daemon mon.b on smithi122 2024-03-11T19:43:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:12.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:12 smithi145 ceph-mon[36730]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:43:12.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:12 smithi145 ceph-mon[36730]: Reconfiguring daemon mon.b on smithi122 2024-03-11T19:43:12.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:12.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:12 smithi107 ceph-mon[31712]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:43:12.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:12 smithi107 ceph-mon[31712]: Reconfiguring daemon mon.b on smithi122 2024-03-11T19:43:12.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:12.568 DEBUG:teuthology.orchestra.run.smithi122:mgr.y> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.y.service 2024-03-11T19:43:12.571 DEBUG:teuthology.orchestra.run.smithi145:mgr.z> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.z.service 2024-03-11T19:43:12.573 INFO:tasks.cephadm:Deploying OSDs... 2024-03-11T19:43:12.573 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:43:12.573 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:43:12.630 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:43:12.630 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-03-11T19:43:12.682 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:43:12.682 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:12.682 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-11T19:43:12.682 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:12.683 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:12.683 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-11 19:42:11.925642631 +0000 2024-03-11T19:43:12.683 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-11 19:36:43.198377599 +0000 2024-03-11T19:43:12.683 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-11 19:36:43.198377599 +0000 2024-03-11T19:43:12.683 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-11 19:36:43.198377599 +0000 2024-03-11T19:43:12.683 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:43:12.772 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-11T19:43:12.772 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-11T19:43:12.772 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000363901 s, 1.4 MB/s 2024-03-11T19:43:12.777 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:43:12.832 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-03-11T19:43:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:43:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:12.878 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 655 Links: 1 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-11 19:42:11.971641866 +0000 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-11 19:36:43.750368239 +0000 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-11 19:36:43.750368239 +0000 2024-03-11T19:43:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-11 19:36:43.750368239 +0000 2024-03-11T19:43:12.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:43:13.011 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-11T19:43:13.011 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-11T19:43:13.011 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00030856 s, 1.7 MB/s 2024-03-11T19:43:13.016 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:43:13.069 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-03-11T19:43:13.115 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:43:13.115 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 673 Links: 1 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-11 19:42:12.017641101 +0000 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-11 19:36:44.396357285 +0000 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-11 19:36:44.396357285 +0000 2024-03-11T19:43:13.116 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-11 19:36:44.396357285 +0000 2024-03-11T19:43:13.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:43:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-11T19:43:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-11T19:43:13.247 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000390515 s, 1.3 MB/s 2024-03-11T19:43:13.251 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:43:13.305 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-03-11T19:43:13.353 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:43:13.353 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:13.353 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-11 19:42:12.078640087 +0000 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-11 19:36:44.950347891 +0000 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-11 19:36:44.951347874 +0000 2024-03-11T19:43:13.354 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-11 19:36:44.950347891 +0000 2024-03-11T19:43:13.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:43:13.439 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-11T19:43:13.439 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-11T19:43:13.439 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000331665 s, 1.5 MB/s 2024-03-11T19:43:13.443 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:43:13.495 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-11T19:43:13.495 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:43:13.514 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:43:13.514 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi122=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: Saving service mgr spec with placement smithi107=x;smithi122=y;smithi145=z;count:3 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:13.528 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:13.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi122=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:13.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: Saving service mgr spec with placement smithi107=x;smithi122=y;smithi145=z;count:3 2024-03-11T19:43:13.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:13.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:13.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:13.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-11 19:42:59.382072337 +0000 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-11 19:35:33.152026943 +0000 2024-03-11T19:43:13.572 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-11 19:35:33.152026943 +0000 2024-03-11T19:43:13.573 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-03-11 19:35:33.152026943 +0000 2024-03-11T19:43:13.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi122=y;smithi145=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: Saving service mgr spec with placement smithi107=x;smithi122=y;smithi145=z;count:3 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:43:13.640 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-11T19:43:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-11T19:43:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000288213 s, 1.8 MB/s 2024-03-11T19:43:13.644 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:43:13.703 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-03-11T19:43:13.760 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-11 19:42:59.382072337 +0000 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-11 19:35:33.498018346 +0000 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-11 19:35:33.498018346 +0000 2024-03-11T19:43:13.761 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-03-11 19:35:33.498018346 +0000 2024-03-11T19:43:13.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:43:13.831 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-11T19:43:13.831 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-11T19:43:13.831 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000273435 s, 1.9 MB/s 2024-03-11T19:43:13.833 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:43:13.892 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-11 19:42:59.383072314 +0000 2024-03-11T19:43:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-11 19:35:33.831010072 +0000 2024-03-11T19:43:13.950 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-11 19:35:33.831010072 +0000 2024-03-11T19:43:13.950 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-03-11 19:35:33.831010072 +0000 2024-03-11T19:43:13.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:43:14.020 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-11T19:43:14.020 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-11T19:43:14.020 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000225175 s, 2.3 MB/s 2024-03-11T19:43:14.022 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:43:14.081 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-11 19:42:59.383072314 +0000 2024-03-11T19:43:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-11 19:35:34.169001673 +0000 2024-03-11T19:43:14.141 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-11 19:35:34.169001673 +0000 2024-03-11T19:43:14.141 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-03-11 19:35:34.169001673 +0000 2024-03-11T19:43:14.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:43:14.209 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-11T19:43:14.209 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-11T19:43:14.209 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000204686 s, 2.5 MB/s 2024-03-11T19:43:14.211 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:43:14.270 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-03-11T19:43:14.270 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:43:14.286 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:43:14.286 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-03-11T19:43:14.342 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-03-11 19:42:49.758382565 +0000 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-03-11 19:35:31.889229881 +0000 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-03-11 19:35:31.889229881 +0000 2024-03-11T19:43:14.343 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-03-11 19:35:31.889229881 +0000 2024-03-11T19:43:14.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:43:14.410 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-03-11T19:43:14.410 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-03-11T19:43:14.410 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000219543 s, 2.3 MB/s 2024-03-11T19:43:14.411 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:43:14.467 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-03-11 19:42:49.758382565 +0000 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-03-11 19:35:32.231221986 +0000 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-03-11 19:35:32.231221986 +0000 2024-03-11T19:43:14.522 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-03-11 19:35:32.231221986 +0000 2024-03-11T19:43:14.523 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:43:14.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: Reconfiguring daemon mon.c on smithi145 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:14.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:14.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:43:14.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: Reconfiguring daemon mon.c on smithi145 2024-03-11T19:43:14.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:14.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:14.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-03-11T19:43:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-03-11T19:43:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000184294 s, 2.8 MB/s 2024-03-11T19:43:14.564 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:43:14.619 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-03-11 19:42:49.758382565 +0000 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-03-11 19:35:32.585213815 +0000 2024-03-11T19:43:14.674 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-03-11 19:35:32.585213815 +0000 2024-03-11T19:43:14.675 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-03-11 19:35:32.585213815 +0000 2024-03-11T19:43:14.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:43:14.739 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-03-11T19:43:14.739 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-03-11T19:43:14.740 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000190055 s, 2.7 MB/s 2024-03-11T19:43:14.741 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:43:14.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: Reconfiguring daemon mon.c on smithi145 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:14.797 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-03-11 19:42:49.759382543 +0000 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-03-11 19:35:32.943205551 +0000 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-03-11 19:35:32.943205551 +0000 2024-03-11T19:43:14.852 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-03-11 19:35:32.943205551 +0000 2024-03-11T19:43:14.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:43:14.918 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-03-11T19:43:14.918 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-03-11T19:43:14.918 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000178114 s, 2.9 MB/s 2024-03-11T19:43:14.919 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:43:14.974 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-03-11T19:43:14.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:43:15.405 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:43:15.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:15.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:15.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:15.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:15.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:15.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:15.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:15 smithi145 ceph-mon[36730]: Deploying daemon mgr.z on smithi145 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:15 smithi107 ceph-mon[31712]: Deploying daemon mgr.z on smithi145 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:15 smithi122 ceph-mon[36397]: Deploying daemon mgr.z on smithi145 2024-03-11T19:43:16.287 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 systemd[1]: Starting Ceph mgr.z for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:43:16.694 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 podman[38588]: 2024-03-11 19:43:16.341589499 +0000 UTC m=+0.017640622 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:43:16.694 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 podman[38588]: 2024-03-11 19:43:16.44498312 +0000 UTC m=+0.121034237 container create 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-11T19:43:16.694 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 podman[38588]: 2024-03-11 19:43:16.664091814 +0000 UTC m=+0.340142946 container init 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-11T19:43:16.695 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 podman[38588]: 2024-03-11 19:43:16.666953958 +0000 UTC m=+0.343005082 container start 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-11T19:43:17.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 ceph-mgr[38607]: -- 172.21.15.145:0/3470652376 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d751f2e1a0 con 0x55d751f04800 2024-03-11T19:43:17.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 bash[38588]: 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 2024-03-11T19:43:17.038 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:16 smithi145 systemd[1]: Started Ceph mgr.z for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:17.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:17.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:17.204+0000 7f08417f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:43:17.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:17.297+0000 7f08417f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:43:17.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:17.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:17.551 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:17.551 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.551 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.551 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:17.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:17.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:43:17.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:17.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:18.270 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:17.929+0000 7f08417f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:43:18.270 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.022+0000 7f08417f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:43:18.270 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.171+0000 7f08417f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0358523 s, 292 MB/s 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-03-11T19:43:18.289 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-11T19:43:18.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-mon[36730]: Deploying daemon mgr.y on smithi122 2024-03-11T19:43:18.538 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.265+0000 7f08417f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:43:18.538 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.350+0000 7f08417f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:43:18.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:18 smithi107 ceph-mon[31712]: Deploying daemon mgr.y on smithi122 2024-03-11T19:43:18.560 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:18 smithi122 ceph-mon[36397]: Deploying daemon mgr.y on smithi122 2024-03-11T19:43:18.908 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.610+0000 7f08417f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:43:18.909 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.719+0000 7f08417f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:43:18.909 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.805+0000 7f08417f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:43:18.909 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:18.904+0000 7f08417f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:43:19.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-03-11T19:43:19.259 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:19.089+0000 7f08417f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:43:19.465 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:43:19.512 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:19 smithi122 ceph-mon[36397]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:19.513 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:19 smithi122 systemd[1]: Started Ceph mgr.y for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:43:19.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:19 smithi145 ceph-mon[36730]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:19.609 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:19 smithi107 ceph-mon[31712]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:20.263 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:19.782+0000 7fc7d0e1d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:43:20.263 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:19.880+0000 7fc7d0e1d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:43:20.390 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:20.103+0000 7f08417f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:43:20.390 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:20.248+0000 7f08417f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:43:20.669 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:20.568+0000 7fc7d0e1d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:43:20.669 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.669 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.669 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.669 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.670 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:20.694 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:20.595+0000 7f08417f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:43:20.694 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:20.689+0000 7f08417f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:43:20.694 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:20.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:20.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:21.001 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:20.664+0000 7fc7d0e1d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:43:21.001 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:20.807+0000 7fc7d0e1d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:43:21.001 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:20.913+0000 7fc7d0e1d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:43:21.001 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:20.996+0000 7fc7d0e1d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:43:21.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:20.887+0000 7f08417f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:43:21.369 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:21.078+0000 7f08417f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:43:21.551 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:21.261+0000 7fc7d0e1d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:43:21.551 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:21.368+0000 7fc7d0e1d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:43:21.551 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:21.453+0000 7fc7d0e1d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:43:21.551 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:21.547+0000 7fc7d0e1d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:21.552 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.555+0000 7ff09f200700 1 -- 172.21.15.107:0/3510503786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100e80 msgr2=0x7ff098101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:21.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.555+0000 7ff09f200700 1 --2- 172.21.15.107:0/3510503786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100e80 0x7ff098101300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff084009990 tx=0x7ff084009ca0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 -- 172.21.15.107:0/3510503786 shutdown_connections 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 --2- 172.21.15.107:0/3510503786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098101840 0x7ff098101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 --2- 172.21.15.107:0/3510503786 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098100520 0x7ff098100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 --2- 172.21.15.107:0/3510503786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100e80 0x7ff098101300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 -- 172.21.15.107:0/3510503786 >> 172.21.15.107:0/3510503786 conn(0x7ff0980fc140 msgr2=0x7ff0980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 -- 172.21.15.107:0/3510503786 shutdown_connections 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.556+0000 7ff09f200700 1 -- 172.21.15.107:0/3510503786 wait complete. 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.557+0000 7ff09f200700 1 Processor -- start 2024-03-11T19:43:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.557+0000 7ff09f200700 1 -- start start 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.557+0000 7ff09f200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.557+0000 7ff09f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 0x7ff09819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff09f200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098101840 0x7ff0981a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff09f200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a1a50 con 0x7ff098100e80 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff09f200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a1b90 con 0x7ff098100520 2024-03-11T19:43:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff09f200700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a1d20 con 0x7ff098101840 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:60708/0 (socket says 172.21.15.107:60708) 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 -- 172.21.15.107:0/2047269494 learned_addr learned my addr 172.21.15.107:0/2047269494 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff09d200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098101840 0x7ff0981a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097400700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 0x7ff09819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 -- 172.21.15.107:0/2047269494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098101840 msgr2=0x7ff0981a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098101840 0x7ff0981a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 -- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 msgr2=0x7ff09819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 0x7ff09819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097e00700 1 -- 172.21.15.107:0/2047269494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840095f0 con 0x7ff098100520 2024-03-11T19:43:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.558+0000 7ff097400700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 0x7ff09819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff097e00700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9a0 tx=0x7ff08800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088009790 con 0x7ff098100520 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff08800cbd0 con 0x7ff098100520 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088018680 con 0x7ff098100520 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0981a1fa0 con 0x7ff098100520 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.559+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0981a2460 con 0x7ff098100520 2024-03-11T19:43:21.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.560+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) v1 ==== 45826+0+0 (secure 0 0 0) 0x7ff088010430 con 0x7ff098100520 2024-03-11T19:43:21.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.560+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09810b300 con 0x7ff098100520 2024-03-11T19:43:21.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.561+0000 7ff094c00700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 0x7ff07c03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:21.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.561+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff088014070 con 0x7ff098100520 2024-03-11T19:43:21.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.561+0000 7ff097400700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 0x7ff07c03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:21.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.561+0000 7ff097400700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 0x7ff07c03b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b2e0 tx=0x7ff08400b7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:21.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.566+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff08801f820 con 0x7ff098100520 2024-03-11T19:43:21.634 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:21.364+0000 7f08417f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:43:21.634 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:21.629+0000 7f08417f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:21.635 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:43:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:43:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:21.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:21.797+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff09802d8d0 con 0x7ff07c038f90 2024-03-11T19:43:22.012 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:21.721+0000 7fc7d0e1d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:43:22.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:21.865+0000 7f08417f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:22.682+0000 7fc7d0e1d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: Reconfiguring daemon mgr.x on smithi107 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-mon[36397]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:22.787 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:22.665+0000 7f08417f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:43:22.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: Reconfiguring daemon mgr.x on smithi107 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:22.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:22 smithi145 ceph-mon[36730]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:22.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:43:22.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: Reconfiguring daemon mgr.x on smithi107 2024-03-11T19:43:22.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:22.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:22.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:22.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:22.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:22 smithi107 ceph-mon[31712]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:23.145 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:22.811+0000 7fc7d0e1d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:43:23.424 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:23.140+0000 7fc7d0e1d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:43:23.424 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:23.228+0000 7fc7d0e1d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:43:23.424 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:23.419+0000 7fc7d0e1d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:43:23.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:23.237+0000 7f08417f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:43:23.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:23.339+0000 7f08417f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:43:23.762 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:23.604+0000 7fc7d0e1d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:43:24.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:43:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:43:23.558+0000 7f08417f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:43:24.141 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:23.881+0000 7fc7d0e1d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:43:24.142 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:24.137+0000 7fc7d0e1d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:43:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:24.409+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 14) v1 ==== 91025+0+0 (secure 0 0 0) 0x7ff08800fd10 con 0x7ff098100520 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:24.372+0000 7fc7d0e1d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: Standby manager daemon z started 2024-03-11T19:43:24.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:43:24.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:24.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: Standby manager daemon z started 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:24.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: Standby manager daemon z started 2024-03-11T19:43:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:43:24.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/1084909501' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:24.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:25.512 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:25.165+0000 7fc7d0e1d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:43:25.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-mon[36397]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:25.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-mon[36397]: mgrmap e14: x(active, since 90s), standbys: z 2024-03-11T19:43:25.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:43:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:25 smithi145 ceph-mon[36730]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:25 smithi145 ceph-mon[36730]: mgrmap e14: x(active, since 90s), standbys: z 2024-03-11T19:43:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:25 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:43:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:25 smithi107 ceph-mon[31712]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:25 smithi107 ceph-mon[31712]: mgrmap e14: x(active, since 90s), standbys: z 2024-03-11T19:43:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:43:26.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:25.728+0000 7fc7d0e1d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:43:26.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:25.823+0000 7fc7d0e1d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:43:26.424 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:43:26.035+0000 7fc7d0e1d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:43:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:26.425+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mon.2 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7ff0880187e0 con 0x7ff098100520 2024-03-11T19:43:26.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:26 smithi107 ceph-mon[31712]: Standby manager daemon y started 2024-03-11T19:43:26.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:26 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:43:26.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:26 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:26.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:26 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:43:26.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:26 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:26.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-mon[36397]: Standby manager daemon y started 2024-03-11T19:43:26.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:43:26.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:26.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:43:26.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:26 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:26.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:26 smithi145 ceph-mon[36730]: Standby manager daemon y started 2024-03-11T19:43:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:26 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:43:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:26 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:43:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:26 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:43:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:26 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/2619176872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]: dispatch 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]': finished 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2545346063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-03-11T19:43:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:43:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]: dispatch 2024-03-11T19:43:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]': finished 2024-03-11T19:43:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:43:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2545346063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-03-11T19:43:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:43:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]: dispatch 2024-03-11T19:43:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2907799133' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa69835-f991-4647-8793-70e0f2472ebb"}]': finished 2024-03-11T19:43:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:43:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2545346063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:29 smithi122 ceph-mon[36397]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:29 smithi145 ceph-mon[36730]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:29 smithi107 ceph-mon[31712]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:31.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:31 smithi122 ceph-mon[36397]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:31 smithi145 ceph-mon[36730]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:31 smithi107 ceph-mon[31712]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:32.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:32 smithi122 ceph-mon[36397]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:32 smithi145 ceph-mon[36730]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:32.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:32 smithi107 ceph-mon[31712]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:35.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:35 smithi107 ceph-mon[31712]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:35.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:35 smithi145 ceph-mon[36730]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:35 smithi122 ceph-mon[36397]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:43:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:36.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:43:36.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:43:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:37.535 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:37 smithi107 ceph-mon[31712]: Deploying daemon osd.0 on smithi107 2024-03-11T19:43:37.535 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:37 smithi107 ceph-mon[31712]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:37.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:37 smithi145 ceph-mon[36730]: Deploying daemon osd.0 on smithi107 2024-03-11T19:43:37.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:37 smithi145 ceph-mon[36730]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:37.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:37 smithi122 ceph-mon[36397]: Deploying daemon osd.0 on smithi107 2024-03-11T19:43:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:37 smithi122 ceph-mon[36397]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:39.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:39 smithi145 ceph-mon[36730]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:39.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:39 smithi107 ceph-mon[31712]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:39 smithi122 ceph-mon[36397]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:41.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:41 smithi145 ceph-mon[36730]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:41.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:41 smithi107 ceph-mon[31712]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:41.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:41 smithi122 ceph-mon[36397]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:43.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:43 smithi107 ceph-mon[31712]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:43 smithi122 ceph-mon[36397]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:43 smithi145 ceph-mon[36730]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:45.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:45.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:44 smithi107 ceph-mon[31712]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:45.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:45.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:44 smithi122 ceph-mon[36397]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:45.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:45.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:45.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:44 smithi145 ceph-mon[36730]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:47.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:47 smithi145 ceph-mon[36730]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:47.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:47 smithi107 ceph-mon[31712]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:47.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:47 smithi122 ceph-mon[36397]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:48.592 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:48.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:48.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:48 smithi107 ceph-mon[31712]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:48 smithi122 ceph-mon[36397]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:43:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:48.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:48.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:48.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:48 smithi145 ceph-mon[36730]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:43:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.836+0000 7ff094c00700 1 -- 172.21.15.107:0/2047269494 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff09802d8d0 con 0x7ff07c038f90 2024-03-11T19:43:48.841 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-03-11T19:43:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 msgr2=0x7ff07c03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 0x7ff07c03b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b2e0 tx=0x7ff08400b7c0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 msgr2=0x7ff098199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9a0 tx=0x7ff08800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 shutdown_connections 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff07c038f90 0x7ff07c03b450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff098101840 0x7ff0981a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff098100520 0x7ff098199f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 --2- 172.21.15.107:0/2047269494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff098100e80 0x7ff09819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 >> 172.21.15.107:0/2047269494 conn(0x7ff0980fc140 msgr2=0x7ff0980fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.841+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 shutdown_connections 2024-03-11T19:43:48.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:43:48.842+0000 7ff09f200700 1 -- 172.21.15.107:0/2047269494 wait complete. 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:49.658 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.0.service 2024-03-11T19:43:49.663 INFO:tasks.cephadm:Deploying osd.1 on smithi122 with /dev/vg_nvme/lv_4... 2024-03-11T19:43:49.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.694 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:49.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:49.906 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:43:50.174 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0[46193]: 2024-03-11T19:43:50.027+0000 7f0d9e000700 -1 osd.0 0 waiting for initial osdmap 2024-03-11T19:43:50.175 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0[46193]: 2024-03-11T19:43:50.040+0000 7f0d93e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:43:50.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:43:50.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:43:50.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' 2024-03-11T19:43:50.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:50 smithi107 ceph-mon[31712]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' 2024-03-11T19:43:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:50 smithi122 ceph-mon[36397]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:43:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:43:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:50.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: from='osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905]' entity='osd.0' 2024-03-11T19:43:50.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:50 smithi145 ceph-mon[36730]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] boot 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:43:51.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] boot 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:43:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:43:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:43:51.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: osd.0 [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] boot 2024-03-11T19:43:51.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:43:51.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:43:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:43:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:43:51.817 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-03-11T19:43:51.817 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-03-11T19:43:51.818 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108956 s, 962 MB/s 2024-03-11T19:43:51.818 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-03-11T19:43:52.412 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-03-11T19:43:52.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:52 smithi107 ceph-mon[31712]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:52.686 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:43:52.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:52 smithi122 ceph-mon[36397]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:52.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:52 smithi145 ceph-mon[36730]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:53.656 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:53 smithi122 ceph-mon[36397]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:43:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:53 smithi145 ceph-mon[36730]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:43:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:53 smithi107 ceph-mon[31712]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:43:54.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 -- 172.21.15.122:0/3439563665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c100ba0 msgr2=0x7eff2c100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 --2- 172.21.15.122:0/3439563665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c100ba0 0x7eff2c100fc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7eff200099c0 tx=0x7eff20009cd0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 -- 172.21.15.122:0/3439563665 shutdown_connections 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 --2- 172.21.15.122:0/3439563665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c102740 0x7eff2c108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 --2- 172.21.15.122:0/3439563665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c100ba0 0x7eff2c100fc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 --2- 172.21.15.122:0/3439563665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c101da0 0x7eff2c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 -- 172.21.15.122:0/3439563665 >> 172.21.15.122:0/3439563665 conn(0x7eff2c0fc160 msgr2=0x7eff2c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 -- 172.21.15.122:0/3439563665 shutdown_connections 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.155+0000 7eff32cbc700 1 -- 172.21.15.122:0/3439563665 wait complete. 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 Processor -- start 2024-03-11T19:43:54.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 -- start start 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 0x7eff2c104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c102740 0x7eff2c1036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c103e40 con 0x7eff2c102740 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c103fb0 con 0x7eff2c101da0 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff32cbc700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1a1de0 con 0x7eff2c100ba0 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48398/0 (socket says 172.21.15.122:48398) 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 -- 172.21.15.122:0/505497181 learned_addr learned my addr 172.21.15.122:0/505497181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff30a58700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 0x7eff2c104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:54.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 -- 172.21.15.122:0/505497181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 msgr2=0x7eff2c104b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 0x7eff2c104b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 -- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c102740 msgr2=0x7eff2c1036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c102740 0x7eff2c1036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff2bfff700 1 -- 172.21.15.122:0/505497181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200095f0 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.156+0000 7eff30a58700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 0x7eff2c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff2bfff700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b530 tx=0x7eff1800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011830 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff18011e70 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c1a20c0 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800f2e0 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.157+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1a2640 con 0x7eff2c101da0 2024-03-11T19:43:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.158+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c068930 con 0x7eff2c101da0 2024-03-11T19:43:54.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.159+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7eff180105f0 con 0x7eff2c101da0 2024-03-11T19:43:54.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.159+0000 7eff29ffb700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 0x7eff1c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:54.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.160+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7eff180cb430 con 0x7eff2c101da0 2024-03-11T19:43:54.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.160+0000 7eff30a58700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 0x7eff1c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:54.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.161+0000 7eff30a58700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 0x7eff1c0a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff200038c0 tx=0x7eff20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:54.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.164+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff180960b0 con 0x7eff2c101da0 2024-03-11T19:43:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:43:54.369+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7eff2c0fe2e0 con 0x7eff1c0a1a30 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi107 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:43:55.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:55.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi107 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi107 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:43:55.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/1786180540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]': finished 2024-03-11T19:43:57.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:43:57.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/1786180540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]': finished 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:43:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:43:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/1786180540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]: dispatch 2024-03-11T19:43:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de9ffc7-d837-48bc-b205-34fe05f25eab"}]': finished 2024-03-11T19:43:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:43:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:43:58.621 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:58 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/876732241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:58.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:58 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/876732241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:58.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:58 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/876732241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:43:59.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:43:59 smithi107 ceph-mon[31712]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:43:59 smithi122 ceph-mon[36397]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:43:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:43:59 smithi145 ceph-mon[36730]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:01.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:01 smithi122 ceph-mon[36397]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:01 smithi145 ceph-mon[36730]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:01 smithi107 ceph-mon[31712]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:03 smithi122 ceph-mon[36397]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:03 smithi145 ceph-mon[36730]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:03 smithi107 ceph-mon[31712]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:04.635 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:44:04.635 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:44:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:04.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:44:04.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:05.512 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:05 smithi122 ceph-mon[36397]: Deploying daemon osd.1 on smithi122 2024-03-11T19:44:05.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:05 smithi122 ceph-mon[36397]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:05.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:05 smithi145 ceph-mon[36730]: Deploying daemon osd.1 on smithi122 2024-03-11T19:44:05.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:05 smithi145 ceph-mon[36730]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:05.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:05 smithi107 ceph-mon[31712]: Deploying daemon osd.1 on smithi122 2024-03-11T19:44:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:05 smithi107 ceph-mon[31712]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:07.483 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:07 smithi122 ceph-mon[36397]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:07.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:07 smithi145 ceph-mon[36730]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:07.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:07 smithi107 ceph-mon[31712]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:09 smithi122 ceph-mon[36397]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:09 smithi145 ceph-mon[36730]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:09 smithi107 ceph-mon[31712]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:11.678 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:11 smithi122 ceph-mon[36397]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:11 smithi145 ceph-mon[36730]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:11 smithi107 ceph-mon[31712]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:12.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.752+0000 7eff29ffb700 1 -- 172.21.15.122:0/505497181 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff2c0fe2e0 con 0x7eff1c0a1a30 2024-03-11T19:44:12.757 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 1 on host 'smithi122' 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 msgr2=0x7eff1c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 0x7eff1c0a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff200038c0 tx=0x7eff20000bc0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 msgr2=0x7eff2c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b530 tx=0x7eff1800b8f0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 shutdown_connections 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff1c0a1a30 0x7eff1c0a3ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c102740 0x7eff2c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff2c100ba0 0x7eff2c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 --2- 172.21.15.122:0/505497181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff2c101da0 0x7eff2c103180 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 >> 172.21.15.122:0/505497181 conn(0x7eff2c0fc160 msgr2=0x7eff2c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 shutdown_connections 2024-03-11T19:44:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:44:12.754+0000 7eff32cbc700 1 -- 172.21.15.122:0/505497181 wait complete. 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:13.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:13.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:13.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:13.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:13.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:13.526 DEBUG:teuthology.orchestra.run.smithi122:osd.1> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.1.service 2024-03-11T19:44:13.529 INFO:tasks.cephadm:Deploying osd.2 on smithi145 with /dev/vg_nvme/lv_4... 2024-03-11T19:44:13.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:44:13.772 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:44:14.070 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:13 smithi122 ceph-mon[36397]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.071 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:13 smithi122 ceph-mon[36397]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.184 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:13 smithi145 ceph-mon[36730]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.184 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:13 smithi145 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:13 smithi107 ceph-mon[31712]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:13 smithi107 ceph-mon[31712]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:44:14.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:44:14.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:14.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:44:14.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:14.949 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:14.950 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:14 smithi145 ceph-mon[36730]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:15.144 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:14 smithi122 ceph-mon[36397]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:15.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:44:15.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: from='osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:15.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:44:15.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:15.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:44:15.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:14 smithi107 ceph-mon[31712]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:15.478 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:44:15.478 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:44:15.479 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-03-11T19:44:15.479 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-03-11T19:44:15.479 INFO:teuthology.orchestra.run.smithi145.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112384 s, 933 MB/s 2024-03-11T19:44:15.479 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 2024-03-11T19:44:15.479 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-03-11T19:44:15.513 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1[40983]: 2024-03-11T19:44:15.339+0000 7fb5c2b79700 -1 osd.1 0 waiting for initial osdmap 2024-03-11T19:44:15.513 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1[40983]: 2024-03-11T19:44:15.343+0000 7fb5ba964700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:44:16.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi145:vg_nvme/lv_4 2024-03-11T19:44:16.180 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-mon[36397]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:44:16.180 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-mon[36397]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:44:16.180 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:16.180 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:15 smithi122 ceph-mon[36397]: from='osd.1 ' entity='osd.1' 2024-03-11T19:44:16.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:15 smithi145 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:44:16.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:15 smithi145 ceph-mon[36730]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:44:16.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:15 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:16.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:15 smithi145 ceph-mon[36730]: from='osd.1 ' entity='osd.1' 2024-03-11T19:44:16.293 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:44:16.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:15 smithi107 ceph-mon[31712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:44:16.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:15 smithi107 ceph-mon[31712]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:44:16.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:15 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:16.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:15 smithi107 ceph-mon[31712]: from='osd.1 ' entity='osd.1' 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] boot 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:17.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:17.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] boot 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:17.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:17.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:17.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:44:17.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: osd.1 [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] boot 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:17.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:17.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/1366085806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78105090 msgr2=0x7f2c78107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/1366085806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78105090 0x7f2c78107480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009990 tx=0x7f2c68009ca0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/1366085806 shutdown_connections 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/1366085806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c781079c0 0x7f2c78109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/1366085806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78102760 0x7f2c78104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/1366085806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78105090 0x7f2c78107480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/1366085806 >> 172.21.15.145:0/1366085806 conn(0x7f2c780fc190 msgr2=0x7f2c780fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/1366085806 shutdown_connections 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/1366085806 wait complete. 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 Processor -- start 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- start start 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78105090 0x7f2c781981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c781079c0 0x7f2c7819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819f800 con 0x7f2c781079c0 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819f940 con 0x7f2c78105090 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.869+0000 7f2c7f7cc700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7819fb00 con 0x7f2c78102760 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56554/0 (socket says 172.21.15.145:56554) 2024-03-11T19:44:17.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 -- 172.21.15.145:0/2169424801 learned_addr learned my addr 172.21.15.145:0/2169424801 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7cd67700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78105090 0x7f2c781981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 -- 172.21.15.145:0/2169424801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78105090 msgr2=0x7f2c781981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78105090 0x7f2c781981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 -- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c781079c0 msgr2=0x7f2c7819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c781079c0 0x7f2c7819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 -- 172.21.15.145:0/2169424801 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680095f0 con 0x7f2c78102760 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7d568700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c920 tx=0x7f2c7400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74007b00 con 0x7f2c78102760 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c7400ce70 con 0x7f2c78102760 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7819fde0 con 0x7f2c78102760 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74005610 con 0x7f2c78102760 2024-03-11T19:44:17.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.870+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c781a0250 con 0x7f2c78102760 2024-03-11T19:44:17.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.871+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c780fdd10 con 0x7f2c78102760 2024-03-11T19:44:17.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.872+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 15) v1 ==== 136224+0+0 (secure 0 0 0) 0x7f2c74004750 con 0x7f2c78102760 2024-03-11T19:44:17.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.873+0000 7f2c6e7fc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 0x7f2c640a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:17.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.873+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f2c740cb9e0 con 0x7f2c78102760 2024-03-11T19:44:17.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.873+0000 7f2c7cd67700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 0x7f2c640a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:17.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.874+0000 7f2c7cd67700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 0x7f2c640a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c78199190 tx=0x7f2c68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:17.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:17.877+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c74096440 con 0x7f2c78102760 2024-03-11T19:44:18.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:18.078+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c7802d950 con 0x7f2c640a1ab0 2024-03-11T19:44:18.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:17 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi122 2024-03-11T19:44:18.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:17 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 17080M 2024-03-11T19:44:18.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:17 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi122 2024-03-11T19:44:18.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:17 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 17080M 2024-03-11T19:44:18.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:17 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi122 2024-03-11T19:44:18.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:17 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 17080M 2024-03-11T19:44:19.216 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:44:19.217 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:19.217 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:19.217 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:19.217 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:19.217 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:18 smithi145 ceph-mon[36730]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:19.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:18 smithi122 ceph-mon[36397]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:19.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:44:19.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:19.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:19.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:19.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:19.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:18 smithi107 ceph-mon[31712]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:21.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:21.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/1566157934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]': finished 2024-03-11T19:44:21.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:44:21.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:21.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:21 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/2474223025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:21.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/1566157934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]': finished 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:21.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:21 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/2474223025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/1566157934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]: dispatch 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cbc8a5-cf58-48bc-8e6e-013627c03c26"}]': finished 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:21 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/2474223025' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:23.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:23 smithi145 ceph-mon[36730]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:23.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:23 smithi107 ceph-mon[31712]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:23 smithi122 ceph-mon[36397]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:25.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:25 smithi145 ceph-mon[36730]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:25.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:25 smithi107 ceph-mon[31712]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:25 smithi122 ceph-mon[36397]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:27 smithi145 ceph-mon[36730]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:27 smithi107 ceph-mon[31712]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:27 smithi122 ceph-mon[36397]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:28.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:44:28.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:44:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:44:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:29.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:29 smithi145 ceph-mon[36730]: Deploying daemon osd.2 on smithi145 2024-03-11T19:44:29.530 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:29 smithi145 ceph-mon[36730]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:29 smithi122 ceph-mon[36397]: Deploying daemon osd.2 on smithi145 2024-03-11T19:44:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:29 smithi122 ceph-mon[36397]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:29 smithi107 ceph-mon[31712]: Deploying daemon osd.2 on smithi145 2024-03-11T19:44:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:29 smithi107 ceph-mon[31712]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:31 smithi122 ceph-mon[36397]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:31.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:31 smithi145 ceph-mon[36730]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:31 smithi107 ceph-mon[31712]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:33.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:33 smithi145 ceph-mon[36730]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:33.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.595 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:33.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:33 smithi122 ceph-mon[36397]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:33.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:33 smithi107 ceph-mon[31712]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:35 smithi122 ceph-mon[36397]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:35.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:35 smithi145 ceph-mon[36730]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:35 smithi107 ceph-mon[31712]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:36.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.113+0000 7f2c6e7fc700 1 -- 172.21.15.145:0/2169424801 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c7802d950 con 0x7f2c640a1ab0 2024-03-11T19:44:36.117 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 2 on host 'smithi145' 2024-03-11T19:44:36.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 msgr2=0x7f2c640a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:36.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 0x7f2c640a3f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c78199190 tx=0x7f2c68005fd0 comp rx=0 tx=0).stop 2024-03-11T19:44:36.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 msgr2=0x7f2c78197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c920 tx=0x7f2c7400cc30 comp rx=0 tx=0).stop 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 shutdown_connections 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c640a1ab0 0x7f2c640a3f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c781079c0 0x7f2c7819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2c78102760 0x7f2c78197c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 --2- 172.21.15.145:0/2169424801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c78105090 0x7f2c781981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 >> 172.21.15.145:0/2169424801 conn(0x7f2c780fc190 msgr2=0x7f2c78103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 shutdown_connections 2024-03-11T19:44:36.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:44:36.116+0000 7f2c7f7cc700 1 -- 172.21.15.145:0/2169424801 wait complete. 2024-03-11T19:44:36.741 DEBUG:teuthology.orchestra.run.smithi145:osd.2> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.2.service 2024-03-11T19:44:36.743 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2024-03-11T19:44:36.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:44:37.017 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T19:44:36.877+0000 7f957a1917c0 -1 osd.2 0 log_to_monitors true 2024-03-11T19:44:37.177 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:44:37.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:37 smithi145 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:37.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:37.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:37.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:37 smithi107 ceph-mon[31712]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:37 smithi122 ceph-mon[36397]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:44:38.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:38 smithi122 ceph-mon[36397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:44:38.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:38 smithi122 ceph-mon[36397]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:38.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:38 smithi122 ceph-mon[36397]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:44:38.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:38 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:38.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:38 smithi122 ceph-mon[36397]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:38.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:44:38.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-mon[36730]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:38.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-mon[36730]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:44:38.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:38.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:38.538 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T19:44:38.218+0000 7f956f7f6700 -1 osd.2 0 waiting for initial osdmap 2024-03-11T19:44:38.538 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T19:44:38.224+0000 7f956b5e9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:44:38.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:38 smithi107 ceph-mon[31712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:44:38.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:38 smithi107 ceph-mon[31712]: from='osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:38.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:38 smithi107 ceph-mon[31712]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:44:38.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:38 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:38.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:38 smithi107 ceph-mon[31712]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:44:39.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:39 smithi107 ceph-mon[31712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:44:39.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:39 smithi107 ceph-mon[31712]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:44:39.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:39.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:39 smithi107 ceph-mon[31712]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:39.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:39.494 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:39 smithi145 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:44:39.494 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:39 smithi145 ceph-mon[36730]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:44:39.494 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:39.494 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:39 smithi145 ceph-mon[36730]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:39.494 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:39.512 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:39 smithi122 ceph-mon[36397]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:44:39.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:39 smithi122 ceph-mon[36397]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:44:39.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:39.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:39 smithi122 ceph-mon[36397]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:44:39.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0326517 s, 321 MB/s 2024-03-11T19:44:40.136 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] boot 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:40.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] boot 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.539 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:40.539 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.539 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.539 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: osd.2 [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] boot 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:44:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:40.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:41.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-03-11T19:44:41.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi145 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 17080M 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:44:41.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi145 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 17080M 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:44:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:44:41.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi145 2024-03-11T19:44:41.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 17080M 2024-03-11T19:44:41.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:41.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:41.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:41.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:44:41.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:44:42.794 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:42 smithi107 sudo[50406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:42.795 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:42 smithi107 sudo[50406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:42.795 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:42 smithi107 sudo[50406]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:42.795 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:42 smithi107 sudo[50406]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:43.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:42 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:43.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:42 smithi107 ceph-mon[31712]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:44:43.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:42 smithi107 ceph-mon[31712]: pgmap v94: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:43.262 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:42 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:43.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:42 smithi122 ceph-mon[36397]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:44:43.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:42 smithi122 ceph-mon[36397]: pgmap v94: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:43.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:42 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:44:43.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:42 smithi145 ceph-mon[36730]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:44:43.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:42 smithi145 ceph-mon[36730]: pgmap v94: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:43.479 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:43 smithi107 sudo[50428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:43.480 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:43 smithi107 sudo[50428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:43.480 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:43 smithi107 sudo[50428]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:43.480 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 19:44:43 smithi107 sudo[50428]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 -- 172.21.15.107:0/786696888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc102ea0 msgr2=0x7f1ddc103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 --2- 172.21.15.107:0/786696888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc103300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009920 tx=0x7f1dc8009c30 comp rx=0 tx=0).stop 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 -- 172.21.15.107:0/786696888 shutdown_connections 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 --2- 172.21.15.107:0/786696888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc103840 0x7f1ddc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 --2- 172.21.15.107:0/786696888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 --2- 172.21.15.107:0/786696888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc103300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 -- 172.21.15.107:0/786696888 >> 172.21.15.107:0/786696888 conn(0x7f1ddc0fd280 msgr2=0x7f1ddc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 -- 172.21.15.107:0/786696888 shutdown_connections 2024-03-11T19:44:43.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.645+0000 7f1de3400700 1 -- 172.21.15.107:0/786696888 wait complete. 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 Processor -- start 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 -- start start 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc103840 0x7f1ddc0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc074d20 con 0x7f1ddc103840 2024-03-11T19:44:43.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc074e90 con 0x7f1ddc102ea0 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.646+0000 7f1de3400700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1a1d70 con 0x7f1ddc101ca0 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1de1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc103840 0x7f1ddc0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40528/0 (socket says 172.21.15.107:40528) 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 -- 172.21.15.107:0/462592525 learned_addr learned my addr 172.21.15.107:0/462592525 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:44:43.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1de0a00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 -- 172.21.15.107:0/462592525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 msgr2=0x7f1ddc075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 -- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc103840 msgr2=0x7f1ddc0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc103840 0x7f1ddc0745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 -- 172.21.15.107:0/462592525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4009d00 con 0x7f1ddc102ea0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1de0a00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1ddbe00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009230 tx=0x7f1dc8005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc801c070 con 0x7f1ddc102ea0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8009520 con 0x7f1ddc102ea0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.647+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc800f910 con 0x7f1ddc102ea0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.648+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8004c00 con 0x7f1ddc102ea0 2024-03-11T19:44:43.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.648+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc1a2270 con 0x7f1ddc102ea0 2024-03-11T19:44:43.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.649+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc068930 con 0x7f1ddc102ea0 2024-03-11T19:44:43.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.650+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1dc8020070 con 0x7f1ddc102ea0 2024-03-11T19:44:43.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.651+0000 7f1dd9600700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 0x7f1dc40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:43.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.652+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f1dc80cccf0 con 0x7f1ddc102ea0 2024-03-11T19:44:43.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.652+0000 7f1de0a00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 0x7f1dc40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:43.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.652+0000 7f1de0a00700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 0x7f1dc40a4000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc102d00 tx=0x7f1dd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:43.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.655+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1dc80972c0 con 0x7f1ddc102ea0 2024-03-11T19:44:43.698 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:43.698 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:43.698 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45148]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:43.698 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45148]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:43.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:44:43.873+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ddc02d910 con 0x7f1dc40a1b40 2024-03-11T19:44:44.012 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45151]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:44.013 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45151]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:44.013 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45151]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:44.013 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 19:44:43 smithi122 sudo[45151]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:44.242 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:44.242 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:43 smithi145 sudo[45809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:44.242 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:43 smithi145 sudo[45809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:44.242 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:43 smithi145 sudo[45809]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:44.243 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:43 smithi145 sudo[45809]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:44.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:44:44.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:44:44.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:44.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:44:44.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:44:44.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:44.537 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:44 smithi145 sudo[45812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:44.537 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:44 smithi145 sudo[45812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:44.538 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:44 smithi145 sudo[45812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:44.538 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 19:44:44 smithi145 sudo[45812]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:45.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:44 smithi107 sudo[50581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:45.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:44 smithi107 sudo[50581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:45.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:44 smithi107 sudo[50581]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:45.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:44 smithi107 sudo[50581]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:45 smithi107 sudo[50601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:45 smithi107 sudo[50601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:45 smithi107 sudo[50601]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:45.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:45 smithi107 sudo[50601]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:45.891 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:45.891 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:45.891 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:45.891 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45815]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:46.148 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:45 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:46.148 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45154]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:44:46.149 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:46.149 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45154]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:46.149 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45154]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:46.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:46.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:46.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:46.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 sudo[45818]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:46.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:45 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:46.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:45 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:44:46.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:44:46.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:44:46.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:44:46.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 sudo[45157]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:44:47.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:47.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: pgmap v96: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: pgmap v97: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:46 smithi107 ceph-mon[31712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:47.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: pgmap v96: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: pgmap v97: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:46 smithi122 ceph-mon[36397]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: pgmap v96: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:44:47.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:44:47.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:44:47.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: pgmap v97: 1 pgs: 1 active+clean; 128 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:47.289 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:46 smithi145 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:44:48.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:47 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]: dispatch 2024-03-11T19:44:48.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:47 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]': finished 2024-03-11T19:44:48.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:47 smithi122 ceph-mon[36397]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:44:48.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:44:48.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:47 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]: dispatch 2024-03-11T19:44:48.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:47 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]': finished 2024-03-11T19:44:48.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:47 smithi145 ceph-mon[36730]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:44:48.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:44:48.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:47 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]: dispatch 2024-03-11T19:44:48.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:47 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/300665270' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec9056e-4b8e-477a-8004-9f802d9b3808"}]': finished 2024-03-11T19:44:48.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:47 smithi107 ceph-mon[31712]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:44:48.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:44:49.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:48 smithi122 ceph-mon[36397]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 468 KiB/s wr, 42 op/s 2024-03-11T19:44:49.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:48 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3603494288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:49.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:48 smithi145 ceph-mon[36730]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 468 KiB/s wr, 42 op/s 2024-03-11T19:44:49.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:48 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3603494288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:49.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:48 smithi107 ceph-mon[31712]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 468 KiB/s wr, 42 op/s 2024-03-11T19:44:49.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:48 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3603494288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:44:51.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:51 smithi145 ceph-mon[36730]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 351 KiB/s wr, 32 op/s 2024-03-11T19:44:51.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:51 smithi107 ceph-mon[31712]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 351 KiB/s wr, 32 op/s 2024-03-11T19:44:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:51 smithi122 ceph-mon[36397]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 351 KiB/s wr, 32 op/s 2024-03-11T19:44:53.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:53 smithi107 ceph-mon[31712]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 301 KiB/s wr, 27 op/s 2024-03-11T19:44:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:53 smithi122 ceph-mon[36397]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 301 KiB/s wr, 27 op/s 2024-03-11T19:44:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:53 smithi145 ceph-mon[36730]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 11 KiB/s rd, 301 KiB/s wr, 27 op/s 2024-03-11T19:44:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:55 smithi107 ceph-mon[31712]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:55.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:55 smithi122 ceph-mon[36397]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:55 smithi145 ceph-mon[36730]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:57.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:57 smithi107 ceph-mon[31712]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:57.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:57 smithi122 ceph-mon[36397]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:57 smithi145 ceph-mon[36730]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 281 KiB/s wr, 25 op/s 2024-03-11T19:44:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:58 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:44:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:58 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:58 smithi107 ceph-mon[31712]: Deploying daemon osd.3 on smithi107 2024-03-11T19:44:58.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:58 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:44:58.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:58 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:58.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:58 smithi122 ceph-mon[36397]: Deploying daemon osd.3 on smithi107 2024-03-11T19:44:58.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:58 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:44:58.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:58 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:58.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:58 smithi145 ceph-mon[36730]: Deploying daemon osd.3 on smithi107 2024-03-11T19:44:59.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:44:59 smithi107 ceph-mon[31712]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:44:59 smithi122 ceph-mon[36397]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:44:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:44:59 smithi145 ceph-mon[36730]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:01.499 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:01 smithi107 ceph-mon[31712]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:01 smithi122 ceph-mon[36397]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:01 smithi145 ceph-mon[36730]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:03.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:03 smithi107 ceph-mon[31712]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:03.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:03 smithi122 ceph-mon[36397]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:03 smithi145 ceph-mon[36730]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:05.590 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:05 smithi145 ceph-mon[36730]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:05.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:05 smithi107 ceph-mon[31712]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:05.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:05 smithi122 ceph-mon[36397]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:07.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:06 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:06 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:07.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:06 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:06 smithi107 ceph-mon[31712]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:06 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:06 smithi122 ceph-mon[36397]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:07.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:06 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:06 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:07.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:06 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:07.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:06 smithi145 ceph-mon[36730]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:09.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:09 smithi107 ceph-mon[31712]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:09 smithi122 ceph-mon[36397]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:09 smithi145 ceph-mon[36730]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:10.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:45:10.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:10.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:10.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.612+0000 7f1dd9600700 1 -- 172.21.15.107:0/462592525 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ddc02d910 con 0x7f1dc40a1b40 2024-03-11T19:45:10.617 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-03-11T19:45:10.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 msgr2=0x7f1dc40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 0x7f1dc40a4000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc102d00 tx=0x7f1dd0009260 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 msgr2=0x7f1ddc074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009230 tx=0x7f1dc8005250 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 shutdown_connections 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1dc40a1b40 0x7f1dc40a4000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ddc103840 0x7f1ddc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ddc101ca0 0x7f1ddc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 --2- 172.21.15.107:0/462592525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ddc102ea0 0x7f1ddc074060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:10.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 >> 172.21.15.107:0/462592525 conn(0x7f1ddc0fd280 msgr2=0x7f1ddc106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 shutdown_connections 2024-03-11T19:45:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:45:10.615+0000 7f1de3400700 1 -- 172.21.15.107:0/462592525 wait complete. 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:45:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:11 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:11.565 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.3.service 2024-03-11T19:45:11.571 INFO:tasks.cephadm:Deploying osd.4 on smithi122 with /dev/vg_nvme/lv_3... 2024-03-11T19:45:11.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.601 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.602 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:11 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:11.791 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:11 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] boot 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:45:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] boot 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:45:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: from='osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: osd.3 [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] boot 2024-03-11T19:45:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:45:12.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:45:13.409 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:13 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:45:13.409 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:13 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:45:13.409 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:13 smithi122 ceph-mon[36397]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:13.409 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:13 smithi122 ceph-mon[36397]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:45:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:13 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:45:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:13 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:45:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:13 smithi145 ceph-mon[36730]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:13 smithi145 ceph-mon[36730]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:45:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:13 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:45:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:13 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:45:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:13 smithi107 ceph-mon[31712]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:45:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:13 smithi107 ceph-mon[31712]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109122 s, 961 MB/s 2024-03-11T19:45:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-03-11T19:45:14.618 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-03-11T19:45:14.896 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:45:15.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:15 smithi107 ceph-mon[31712]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:15.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:15 smithi107 ceph-mon[31712]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:45:15.690 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:15 smithi122 ceph-mon[36397]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:15.690 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:15 smithi122 ceph-mon[36397]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:45:15.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:15 smithi145 ceph-mon[36730]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:15.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:15 smithi145 ceph-mon[36730]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/2997443278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44101ec0 msgr2=0x7f1b44108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/2997443278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44101ec0 0x7f1b44108770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b0a0 tx=0x7f1b3c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/2997443278 shutdown_connections 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/2997443278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44101ec0 0x7f1b44108770 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/2997443278 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44100320 0x7f1b44100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/2997443278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101520 0x7f1b44101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/2997443278 >> 172.21.15.122:0/2997443278 conn(0x7f1b440fb8e0 msgr2=0x7f1b440fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/2997443278 shutdown_connections 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.576+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/2997443278 wait complete. 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 Processor -- start 2024-03-11T19:45:16.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 -- start start 2024-03-11T19:45:16.595 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:16 smithi122 ceph-mon[36397]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:45:16.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:16.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44101520 0x7f1b441982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:16.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101ec0 0x7f1b4419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:16.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419f8f0 con 0x7f1b44101ec0 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419fa30 con 0x7f1b44100320 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b4a8d4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4419fbc0 con 0x7f1b44101520 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41230/0 (socket says 172.21.15.122:41230) 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 -- 172.21.15.122:0/3843317083 learned_addr learned my addr 172.21.15.122:0/3843317083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b48e71700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101ec0 0x7f1b4419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 -- 172.21.15.122:0/3843317083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44101520 msgr2=0x7f1b441982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b3bfff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44101520 0x7f1b441982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44101520 0x7f1b441982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 -- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101ec0 msgr2=0x7f1b4419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101ec0 0x7f1b4419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:16.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.577+0000 7f1b43fff700 1 -- 172.21.15.122:0/3843317083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c009d00 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b43fff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c970 tx=0x7f1b3400cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34004220 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b34007d80 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4419fea0 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34003da0 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.578+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b441a0330 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.579+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44107680 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.580+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1b340078a0 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.580+0000 7f1b41ffb700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 0x7f1b300a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.580+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f1b340cc200 con 0x7f1b44100320 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.581+0000 7f1b3bfff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 0x7f1b300a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.581+0000 7f1b3bfff700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 0x7f1b300a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:16.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.584+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b34096630 con 0x7f1b44100320 2024-03-11T19:45:16.624 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:16 smithi107 ceph-mon[31712]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:45:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:16 smithi145 ceph-mon[36730]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:45:16.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:16.785+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b4402da90 con 0x7f1b300a1ac0 2024-03-11T19:45:17.664 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:17 smithi122 ceph-mon[36397]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:17.664 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:17 smithi122 ceph-mon[36397]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:45:17.664 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:17.664 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:17.664 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:17 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:17 smithi145 ceph-mon[36730]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:17 smithi145 ceph-mon[36730]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:45:17.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:17.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:17.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:17 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:17 smithi107 ceph-mon[31712]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:17 smithi107 ceph-mon[31712]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:45:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:17.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:17 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:18.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:18.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:18.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:18.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:18.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:18.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:19.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:19 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi107 2024-03-11T19:45:19.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:19 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-11T19:45:19.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:19 smithi122 ceph-mon[36397]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:45:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:19 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi107 2024-03-11T19:45:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:19 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-11T19:45:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:19 smithi145 ceph-mon[36730]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:45:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:19 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi107 2024-03-11T19:45:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:19 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-11T19:45:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:19 smithi107 ceph-mon[31712]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/2984359044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]': finished 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:20.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:20 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/2980015100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/2984359044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]': finished 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:20 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/2980015100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:20.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/2984359044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]: dispatch 2024-03-11T19:45:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafc0f8b-5df2-4a3e-86de-88558c60282e"}]': finished 2024-03-11T19:45:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:45:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:20 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/2980015100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:21.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:21 smithi122 ceph-mon[36397]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:45:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:21 smithi145 ceph-mon[36730]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:45:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:21 smithi107 ceph-mon[31712]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:45:23.610 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:23 smithi122 ceph-mon[36397]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-11T19:45:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:23 smithi145 ceph-mon[36730]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-11T19:45:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:23 smithi107 ceph-mon[31712]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-11T19:45:25.615 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:25 smithi122 ceph-mon[36397]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:45:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:25 smithi145 ceph-mon[36730]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:45:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:25 smithi107 ceph-mon[31712]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:45:27.628 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:27 smithi107 ceph-mon[31712]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:45:27.628 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:45:27.628 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:27.631 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:27 smithi122 ceph-mon[36397]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:45:27.631 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:45:27.632 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:27 smithi145 ceph-mon[36730]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:45:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:45:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:28.481 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:28 smithi122 ceph-mon[36397]: Deploying daemon osd.4 on smithi122 2024-03-11T19:45:28.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:28 smithi107 ceph-mon[31712]: Deploying daemon osd.4 on smithi122 2024-03-11T19:45:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:28 smithi145 ceph-mon[36730]: Deploying daemon osd.4 on smithi122 2024-03-11T19:45:29.471 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:29 smithi122 ceph-mon[36397]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:29.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:29 smithi107 ceph-mon[31712]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:29 smithi145 ceph-mon[36730]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:31.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:31 smithi122 ceph-mon[36397]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:31 smithi145 ceph-mon[36730]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:31 smithi107 ceph-mon[31712]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:33.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:33.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:33 smithi145 ceph-mon[36730]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:33 smithi107 ceph-mon[31712]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:33.512 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:33 smithi122 ceph-mon[36397]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:35.418 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:35 smithi122 ceph-mon[36397]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:35.470 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 4 on host 'smithi122' 2024-03-11T19:45:35.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.465+0000 7f1b41ffb700 1 -- 172.21.15.122:0/3843317083 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b4402da90 con 0x7f1b300a1ac0 2024-03-11T19:45:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 msgr2=0x7f1b300a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 0x7f1b300a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c008040 comp rx=0 tx=0).stop 2024-03-11T19:45:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 msgr2=0x7f1b44197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400c970 tx=0x7f1b3400cd30 comp rx=0 tx=0).stop 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 shutdown_connections 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1b300a1ac0 0x7f1b300a3f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b44101ec0 0x7f1b4419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b44100320 0x7f1b44197d60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 --2- 172.21.15.122:0/3843317083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1b44101520 0x7f1b441982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 >> 172.21.15.122:0/3843317083 conn(0x7f1b440fb8e0 msgr2=0x7f1b441047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 shutdown_connections 2024-03-11T19:45:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:45:35.468+0000 7f1b4a8d4700 1 -- 172.21.15.122:0/3843317083 wait complete. 2024-03-11T19:45:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:35 smithi145 ceph-mon[36730]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:35 smithi107 ceph-mon[31712]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:36.160 DEBUG:teuthology.orchestra.run.smithi122:osd.4> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.4.service 2024-03-11T19:45:36.162 INFO:tasks.cephadm:Deploying osd.5 on smithi145 with /dev/vg_nvme/lv_3... 2024-03-11T19:45:36.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:45:36.408 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:45:36.716 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.717 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.717 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.717 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.717 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:36.717 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:36 smithi145 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:36.754 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:36 smithi122 ceph-mon[36397]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:36 smithi107 ceph-mon[31712]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:37.745 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:37 smithi145 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:37.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:45:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:45:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-mon[36397]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:45:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: from='osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:45:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:37 smithi107 ceph-mon[31712]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:45:38.263 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T19:45:37.821+0000 7f86e1336700 -1 osd.4 0 waiting for initial osdmap 2024-03-11T19:45:38.263 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 19:45:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T19:45:37.826+0000 7f86dd129700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113708 s, 922 MB/s 2024-03-11T19:45:38.434 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-03-11T19:45:39.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:38 smithi122 ceph-mon[36397]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:45:39.536 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:38 smithi122 ceph-mon[36397]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:45:39.536 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:38 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:39.536 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:38 smithi122 ceph-mon[36397]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:38 smithi107 ceph-mon[31712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:38 smithi107 ceph-mon[31712]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:38 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:38 smithi107 ceph-mon[31712]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:38 smithi145 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:38 smithi145 ceph-mon[36730]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:45:39.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:38 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:39.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:38 smithi145 ceph-mon[36730]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:45:39.538 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi145:vg_nvme/lv_3 2024-03-11T19:45:39.777 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:45:40.083 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:45:40.083 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:45:40.083 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] boot 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi122 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 8540M 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.084 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:40.085 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.085 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.085 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:45:40.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] boot 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi122 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.265 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 8540M 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.266 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.267 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.267 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:40.267 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.267 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.267 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: osd.4 [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] boot 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi122 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 8540M 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:40.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:41.147 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:40 smithi145 ceph-mon[36730]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:45:41.147 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:40 smithi145 ceph-mon[36730]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:41.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:40 smithi122 ceph-mon[36397]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:45:41.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:40 smithi122 ceph-mon[36397]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:41.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:40 smithi107 ceph-mon[31712]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:45:41.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:40 smithi107 ceph-mon[31712]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:41.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/2770151174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4102720 msgr2=0x7f3bf4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:41.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/2770151174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4102720 0x7f3bf4104b10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/2770151174 shutdown_connections 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/2770151174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 0x7f3bf4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/2770151174 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4102720 0x7f3bf4104b10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/2770151174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4105050 0x7f3bf4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/2770151174 >> 172.21.15.145:0/2770151174 conn(0x7f3bf40fc170 msgr2=0x7f3bf40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/2770151174 shutdown_connections 2024-03-11T19:45:41.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.336+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/2770151174 wait complete. 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 Processor -- start 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 -- start start 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4102720 0x7f3bf4197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 0x7f3bf419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:41.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419f820 con 0x7f3bf4102720 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419f960 con 0x7f3bf4107980 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf8a9c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419fb20 con 0x7f3bf4105050 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46126/0 (socket says 172.21.15.145:46126) 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 -- 172.21.15.145:0/53987148 learned_addr learned my addr 172.21.15.145:0/53987148 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf2d9d700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 0x7f3bf419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf259c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4102720 0x7f3bf4197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 -- 172.21.15.145:0/53987148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 msgr2=0x7f3bf419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 0x7f3bf419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 -- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4102720 msgr2=0x7f3bf4197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4102720 0x7f3bf4197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 -- 172.21.15.145:0/53987148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40095f0 con 0x7f3bf4105050 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3bf1d9b700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3be800b530 tx=0x7f3be800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.337+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8011830 con 0x7f3bf4105050 2024-03-11T19:45:41.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.338+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3be8011e70 con 0x7f3bf4105050 2024-03-11T19:45:41.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.338+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf419fe00 con 0x7f3bf4105050 2024-03-11T19:45:41.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.338+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800f2e0 con 0x7f3bf4105050 2024-03-11T19:45:41.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.338+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf41a0270 con 0x7f3bf4105050 2024-03-11T19:45:41.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.339+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4068930 con 0x7f3bf4105050 2024-03-11T19:45:41.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.340+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3be80105f0 con 0x7f3bf4105050 2024-03-11T19:45:41.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.341+0000 7f3be37fe700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 0x7f3bdc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:41.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.341+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f3be80cc2e0 con 0x7f3bf4105050 2024-03-11T19:45:41.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.341+0000 7f3bf259c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 0x7f3bdc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:41.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.342+0000 7f3bf259c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 0x7f3bdc0a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b5c0 tx=0x7f3be4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:41.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.345+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3be8096560 con 0x7f3bf4105050 2024-03-11T19:45:41.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:45:41.545+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bf402d910 con 0x7f3bdc0a1ac0 2024-03-11T19:45:42.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:42.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:42.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:42.262 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:42.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:42.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:42.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:45:42.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:45:42.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:42.996 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:42 smithi145 ceph-mon[36730]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:42.996 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:42 smithi145 ceph-mon[36730]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:43.262 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:42 smithi122 ceph-mon[36397]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:43.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:42 smithi122 ceph-mon[36397]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:43.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:42 smithi107 ceph-mon[31712]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:45:43.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:42 smithi107 ceph-mon[31712]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:44.453 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:44 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/2745806360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.453 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:44 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.453 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:44 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]': finished 2024-03-11T19:45:44.453 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:44 smithi145 ceph-mon[36730]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:45:44.453 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:45:44.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:44 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/2745806360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:44 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:44 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]': finished 2024-03-11T19:45:44.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:44 smithi122 ceph-mon[36397]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:45:44.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:45:44.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:44 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/2745806360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:44 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]: dispatch 2024-03-11T19:45:44.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:44 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3144ef43-abab-4f02-b28e-6ac842870975"}]': finished 2024-03-11T19:45:44.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:44 smithi107 ceph-mon[31712]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:45:44.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:45:45.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:45 smithi122 ceph-mon[36397]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:45.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:45 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/2961022598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:45.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:45 smithi145 ceph-mon[36730]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:45.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:45 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/2961022598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:45 smithi107 ceph-mon[31712]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:45 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/2961022598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:45:47.590 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:47 smithi122 ceph-mon[36397]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:47.736 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:47 smithi145 ceph-mon[36730]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:47.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:47 smithi107 ceph-mon[31712]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:49 smithi122 ceph-mon[36397]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:49.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:49 smithi145 ceph-mon[36730]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:49 smithi107 ceph-mon[31712]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:51.573 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:51 smithi107 ceph-mon[31712]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:51.573 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:45:51.574 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:51 smithi122 ceph-mon[36397]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:45:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:51 smithi145 ceph-mon[36730]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:45:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:52.623 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:52 smithi145 ceph-mon[36730]: Deploying daemon osd.5 on smithi145 2024-03-11T19:45:52.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:52 smithi122 ceph-mon[36397]: Deploying daemon osd.5 on smithi145 2024-03-11T19:45:52.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:52 smithi107 ceph-mon[31712]: Deploying daemon osd.5 on smithi145 2024-03-11T19:45:53.613 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:53 smithi145 ceph-mon[36730]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:53 smithi122 ceph-mon[36397]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:53 smithi107 ceph-mon[31712]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:55.630 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:55 smithi145 ceph-mon[36730]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:55.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:55 smithi122 ceph-mon[36397]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:55 smithi107 ceph-mon[31712]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:57 smithi145 ceph-mon[36730]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:57.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:57 smithi107 ceph-mon[31712]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:57.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:57 smithi122 ceph-mon[36397]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:45:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:59.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:45:59 smithi107 ceph-mon[31712]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:59.675 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:45:59 smithi145 ceph-mon[36730]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:45:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:45:59 smithi122 ceph-mon[36397]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:00.433 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 5 on host 'smithi145' 2024-03-11T19:46:00.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.428+0000 7f3be37fe700 1 -- 172.21.15.145:0/53987148 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3bf402d910 con 0x7f3bdc0a1ac0 2024-03-11T19:46:00.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 msgr2=0x7f3bdc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:00.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 0x7f3bdc0a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b5c0 tx=0x7f3be4005fb0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 msgr2=0x7f3bf4198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:00.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3be800b530 tx=0x7f3be800b8f0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 shutdown_connections 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3bdc0a1ac0 0x7f3bdc0a3f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf4107980 0x7f3bf419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bf4102720 0x7f3bf4197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.430+0000 7f3bf8a9c700 1 --2- 172.21.15.145:0/53987148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bf4105050 0x7f3bf4198250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.431+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 >> 172.21.15.145:0/53987148 conn(0x7f3bf40fc170 msgr2=0x7f3bf41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.431+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 shutdown_connections 2024-03-11T19:46:00.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:46:00.431+0000 7f3bf8a9c700 1 -- 172.21.15.145:0/53987148 wait complete. 2024-03-11T19:46:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:00.746 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.747 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:00 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:00 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:00 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:01.080 DEBUG:teuthology.orchestra.run.smithi145:osd.5> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.5.service 2024-03-11T19:46:01.082 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-03-11T19:46:01.082 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:46:01.524 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:46:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:01 smithi145 ceph-mon[36730]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:01.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:01 smithi145 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:01 smithi107 ceph-mon[31712]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:01 smithi107 ceph-mon[31712]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:02.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:01 smithi122 ceph-mon[36397]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:02.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:01 smithi122 ceph-mon[36397]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:46:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:46:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:46:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-mon[36730]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:46:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:46:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:02 smithi107 ceph-mon[31712]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: from='osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:46:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:02 smithi122 ceph-mon[36397]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:46:03.288 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 19:46:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T19:46:02.993+0000 7f34c20b1700 -1 osd.5 0 waiting for initial osdmap 2024-03-11T19:46:03.288 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T19:46:03.000+0000 7f34b9e9c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:46:03.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:46:03.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:46:03.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:46:03.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:46:03.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:03.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:03.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: from='osd.5 ' entity='osd.5' 2024-03-11T19:46:03.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:03 smithi107 ceph-mon[31712]: osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] boot 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: from='osd.5 ' entity='osd.5' 2024-03-11T19:46:03.752 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:03 smithi145 ceph-mon[36730]: osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] boot 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: from='osd.5 ' entity='osd.5' 2024-03-11T19:46:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:03 smithi122 ceph-mon[36397]: osd.5 [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] boot 2024-03-11T19:46:04.588 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:46:04.588 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:46:04.588 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-11T19:46:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-11T19:46:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343024 s, 306 MB/s 2024-03-11T19:46:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-11T19:46:04.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi145 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 8540M 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:04.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:04 smithi107 ceph-mon[31712]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi145 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 8540M 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:05.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:05.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:04 smithi122 ceph-mon[36397]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:46:05.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi145 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 8540M 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:05.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:05.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:04 smithi145 ceph-mon[36730]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:46:05.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-03-11T19:46:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:46:06.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:06 smithi145 ceph-mon[36730]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:46:06.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:06 smithi145 ceph-mon[36730]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:06.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:06 smithi107 ceph-mon[31712]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:46:06.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:06 smithi107 ceph-mon[31712]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:07.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:06 smithi122 ceph-mon[36397]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:46:07.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:06 smithi122 ceph-mon[36397]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:07.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:07 smithi107 ceph-mon[31712]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:46:08.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:07 smithi122 ceph-mon[36397]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:46:08.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:07 smithi145 ceph-mon[36730]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 -- 172.21.15.107:0/3480305477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe4109d80 msgr2=0x7fabe4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 --2- 172.21.15.107:0/3480305477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe4109d80 0x7fabe4110630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009fc0 tx=0x7fabdc00b330 comp rx=0 tx=0).stop 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 -- 172.21.15.107:0/3480305477 shutdown_connections 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 --2- 172.21.15.107:0/3480305477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe4109d80 0x7fabe4110630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 --2- 172.21.15.107:0/3480305477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe4108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 --2- 172.21.15.107:0/3480305477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe41093e0 0x7fabe4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 -- 172.21.15.107:0/3480305477 >> 172.21.15.107:0/3480305477 conn(0x7fabe40771b0 msgr2=0x7fabe40795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 -- 172.21.15.107:0/3480305477 shutdown_connections 2024-03-11T19:46:08.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.486+0000 7fabe9600700 1 -- 172.21.15.107:0/3480305477 wait complete. 2024-03-11T19:46:08.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.487+0000 7fabe9600700 1 Processor -- start 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.487+0000 7fabe9600700 1 -- start start 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe41093e0 0x7fabe419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 0x7fabe41a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41a5dc0 con 0x7fabe4109d80 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41a5f00 con 0x7fabe41093e0 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe9600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41a6090 con 0x7fabe41081e0 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 0x7fabe41a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 0x7fabe41a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60128/0 (socket says 172.21.15.107:60128) 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe41093e0 0x7fabe419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:08.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe8c00700 1 -- 172.21.15.107:0/1615105005 learned_addr learned my addr 172.21.15.107:0/1615105005 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 -- 172.21.15.107:0/1615105005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe41093e0 msgr2=0x7fabe419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe41093e0 0x7fabe419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 -- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 msgr2=0x7fabe41a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 0x7fabe41a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.488+0000 7fabe3e00700 1 -- 172.21.15.107:0/1615105005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00095f0 con 0x7fabe41081e0 2024-03-11T19:46:08.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe3e00700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe419e230 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fabd400e9f0 tx=0x7fabd400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004220 con 0x7fabe41081e0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc009c40 con 0x7fabe41081e0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabd4004380 con 0x7fabe41081e0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe41a6730 con 0x7fabe41081e0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.489+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4003c20 con 0x7fabe41081e0 2024-03-11T19:46:08.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.490+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe410f5a0 con 0x7fabe41081e0 2024-03-11T19:46:08.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.492+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fabd400c7c0 con 0x7fabe41081e0 2024-03-11T19:46:08.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.494+0000 7fabe0c00700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 0x7fabc80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:08.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.494+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fabd4014070 con 0x7fabe41081e0 2024-03-11T19:46:08.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.494+0000 7fabe3400700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 0x7fabc80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:08.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.495+0000 7fabe3400700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 0x7fabc80a4000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005fd0 tx=0x7fabd0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:08.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.496+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabd40968c0 con 0x7fabe41081e0 2024-03-11T19:46:08.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:08.727+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fabe40620b0 con 0x7fabc80a1b40 2024-03-11T19:46:08.733 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:08 smithi107 ceph-mon[31712]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:46:08.733 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:08 smithi107 ceph-mon[31712]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:46:09.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:08 smithi122 ceph-mon[36397]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:46:09.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:08 smithi122 ceph-mon[36397]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:46:09.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:08 smithi145 ceph-mon[36730]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:46:09.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:08 smithi145 ceph-mon[36730]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:46:09.865 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:09 smithi107 ceph-mon[31712]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:10.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:10.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:10.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:10.120 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:09 smithi145 ceph-mon[36730]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:10.120 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:09 smithi122 ceph-mon[36397]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:10.121 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:11.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:10 smithi122 ceph-mon[36397]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-11T19:46:11.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:10 smithi145 ceph-mon[36730]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-11T19:46:11.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:10 smithi107 ceph-mon[31712]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-11T19:46:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-11T19:46:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2339295660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]': finished 2024-03-11T19:46:13.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:46:13.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-11T19:46:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2339295660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]': finished 2024-03-11T19:46:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:46:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2339295660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]: dispatch 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474c8a3c-29ec-45d8-8580-952d3bce081a"}]': finished 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:46:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:14 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/740829377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:14.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:14 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/740829377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:14.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:14 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/740829377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:15 smithi122 ceph-mon[36397]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:46:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:15 smithi145 ceph-mon[36730]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:46:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:15 smithi107 ceph-mon[31712]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:46:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:17 smithi107 ceph-mon[31712]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:17 smithi122 ceph-mon[36397]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:17 smithi145 ceph-mon[36730]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:19.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:19 smithi107 ceph-mon[31712]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:19 smithi122 ceph-mon[36397]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:19 smithi145 ceph-mon[36730]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:21.623 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:21 smithi107 ceph-mon[31712]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:21 smithi122 ceph-mon[36397]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:21 smithi145 ceph-mon[36730]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:23.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:23 smithi107 ceph-mon[31712]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:23.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:23 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:46:23.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:23 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:23.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:23 smithi122 ceph-mon[36397]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:23 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:46:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:23 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:23 smithi145 ceph-mon[36730]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:23 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:46:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:23 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:24.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:24 smithi107 ceph-mon[31712]: Deploying daemon osd.6 on smithi107 2024-03-11T19:46:24.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:24 smithi122 ceph-mon[36397]: Deploying daemon osd.6 on smithi107 2024-03-11T19:46:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:24 smithi145 ceph-mon[36730]: Deploying daemon osd.6 on smithi107 2024-03-11T19:46:25.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:25 smithi107 ceph-mon[31712]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:25.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:25 smithi122 ceph-mon[36397]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:25 smithi145 ceph-mon[36730]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:27.651 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:27 smithi107 ceph-mon[31712]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:27 smithi122 ceph-mon[36397]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:27 smithi145 ceph-mon[36730]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:29 smithi122 ceph-mon[36397]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:29 smithi145 ceph-mon[36730]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:29.792 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:29 smithi107 ceph-mon[31712]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:31 smithi122 ceph-mon[36397]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:31 smithi145 ceph-mon[36730]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:31 smithi107 ceph-mon[31712]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:32.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.744 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:32.744 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:32.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:32.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:33.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:33 smithi122 ceph-mon[36397]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:33.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:33 smithi107 ceph-mon[31712]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:33.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:33 smithi145 ceph-mon[36730]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:35 smithi107 ceph-mon[31712]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:35 smithi122 ceph-mon[36397]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:35 smithi145 ceph-mon[36730]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:36 smithi122 ceph-mon[36397]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:46:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:36.786 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:36 smithi145 ceph-mon[36730]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:46:36.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:36.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:36 smithi107 ceph-mon[31712]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:46:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.083 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-03-11T19:46:37.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.077+0000 7fabe0c00700 1 -- 172.21.15.107:0/1615105005 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fabe40620b0 con 0x7fabc80a1b40 2024-03-11T19:46:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 msgr2=0x7fabc80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 0x7fabc80a4000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005fd0 tx=0x7fabd0009310 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 msgr2=0x7fabe419e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe419e230 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fabd400e9f0 tx=0x7fabd400ed00 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 shutdown_connections 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fabc80a1b40 0x7fabc80a4000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe4109d80 0x7fabe41a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fabe41081e0 0x7fabe419e230 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 --2- 172.21.15.107:0/1615105005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe41093e0 0x7fabe419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 >> 172.21.15.107:0/1615105005 conn(0x7fabe40771b0 msgr2=0x7fabe410c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 shutdown_connections 2024-03-11T19:46:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:46:37.081+0000 7fabe9600700 1 -- 172.21.15.107:0/1615105005 wait complete. 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:46:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:46:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:46:37.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:37.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:46:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:46:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:46:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:38.104 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.6.service 2024-03-11T19:46:38.109 INFO:tasks.cephadm:Deploying osd.7 on smithi122 with /dev/vg_nvme/lv_2... 2024-03-11T19:46:38.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:46:38.361 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:46:38.607 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:38 smithi107 ceph-mon[31712]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:46:38.607 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:38 smithi107 ceph-mon[31712]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:46:38.607 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:38 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:38.607 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:38 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:38.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:38 smithi122 ceph-mon[36397]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:46:38.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:38 smithi122 ceph-mon[36397]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:46:38.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:38 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:38.702 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:38 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:38 smithi145 ceph-mon[36730]: from='osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:46:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:38 smithi145 ceph-mon[36730]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:46:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:38 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:38 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] boot 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:46:39.710 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:46:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:46:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] boot 2024-03-11T19:46:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:46:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: osd.6 [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] boot 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:46:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108724 s, 964 MB/s 2024-03-11T19:46:40.262 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-03-11T19:46:40.614 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:40 smithi122 ceph-mon[36397]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:46:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:40 smithi145 ceph-mon[36730]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:46:40.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:40 smithi107 ceph-mon[31712]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:46:40.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-03-11T19:46:41.192 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:46:41.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:41 smithi122 ceph-mon[36397]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:41 smithi145 ceph-mon[36730]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:41.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:41 smithi107 ceph-mon[31712]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 -- 172.21.15.122:0/3025191402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 msgr2=0x7f7b181022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/3025191402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b181022a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0099c0 tx=0x7f7b0c009cd0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 -- 172.21.15.122:0/3025191402 shutdown_connections 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/3025191402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b181027e0 0x7f7b18109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/3025191402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b18100c40 0x7f7b18101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/3025191402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b181022a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 -- 172.21.15.122:0/3025191402 >> 172.21.15.122:0/3025191402 conn(0x7f7b180fc200 msgr2=0x7f7b180fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:42.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 -- 172.21.15.122:0/3025191402 shutdown_connections 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.783+0000 7f7b1e86e700 1 -- 172.21.15.122:0/3025191402 wait complete. 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 Processor -- start 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 -- start start 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b18100c40 0x7f7b1810b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b181027e0 0x7f7b1810a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1810a860 con 0x7f7b181027e0 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1810a9d0 con 0x7f7b18101e40 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b1e86e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b181a1d50 con 0x7f7b18100c40 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b177fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b177fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50164/0 (socket says 172.21.15.122:50164) 2024-03-11T19:46:42.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b177fe700 1 -- 172.21.15.122:0/1451214598 learned_addr learned my addr 172.21.15.122:0/1451214598 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.784+0000 7f7b17fff700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b18100c40 0x7f7b1810b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 -- 172.21.15.122:0/1451214598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b18100c40 msgr2=0x7f7b1810b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b18100c40 0x7f7b1810b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 -- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b181027e0 msgr2=0x7f7b1810a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b181027e0 0x7f7b1810a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 -- 172.21.15.122:0/1451214598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0095f0 con 0x7f7b18101e40 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b177fe700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c003790 tx=0x7f7b0c005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c01d070 con 0x7f7b18101e40 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b181a1fd0 con 0x7f7b18101e40 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c004110 con 0x7f7b18101e40 2024-03-11T19:46:42.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c022da0 con 0x7f7b18101e40 2024-03-11T19:46:42.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.785+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b181a2440 con 0x7f7b18101e40 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.786+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b18108080 con 0x7f7b18101e40 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.787+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7b0c022470 con 0x7f7b18101e40 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.788+0000 7f7b157fa700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 0x7f7b000a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.788+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f7b0c0cdbe0 con 0x7f7b18101e40 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.788+0000 7f7b17fff700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 0x7f7b000a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:42.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.789+0000 7f7b17fff700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 0x7f7b000a3f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7b18101ca0 tx=0x7f7b08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:42.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.791+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b0c097b90 con 0x7f7b18101e40 2024-03-11T19:46:42.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:46:42.992+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b1802d8e0 con 0x7f7b000a1ac0 2024-03-11T19:46:43.743 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:43.743 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:43.743 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:43.743 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.743 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:43.744 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi107 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:44.728 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:44 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:44.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:44.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi107 2024-03-11T19:46:44.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-11T19:46:44.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:44.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:44.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:44 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi107 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:44.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:44 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:45 smithi122 ceph-mon[36397]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:45 smithi145 ceph-mon[36730]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:45.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:45 smithi107 ceph-mon[31712]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:46.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/1349836937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]': finished 2024-03-11T19:46:46.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:46:46.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:46:46.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:46 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/2933175611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:46.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/1349836937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]': finished 2024-03-11T19:46:46.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:46:46.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:46:46.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:46 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/2933175611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/1349836937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]: dispatch 2024-03-11T19:46:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ddf2101-a45c-448e-8caa-fab9b4d3bb2f"}]': finished 2024-03-11T19:46:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:46:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:46:46.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:46 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/2933175611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:46:47.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:47 smithi122 ceph-mon[36397]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:47 smithi145 ceph-mon[36730]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:47.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:47 smithi107 ceph-mon[31712]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:49.662 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:49 smithi122 ceph-mon[36397]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:49 smithi145 ceph-mon[36730]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:49 smithi107 ceph-mon[31712]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:51.639 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:51 smithi122 ceph-mon[36397]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:51 smithi145 ceph-mon[36730]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:51.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:51 smithi107 ceph-mon[31712]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:53.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:53 smithi122 ceph-mon[36397]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:53.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:46:53.703 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:53 smithi145 ceph-mon[36730]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:46:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:53 smithi107 ceph-mon[31712]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:46:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:54.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:54 smithi122 ceph-mon[36397]: Deploying daemon osd.7 on smithi122 2024-03-11T19:46:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:54 smithi145 ceph-mon[36730]: Deploying daemon osd.7 on smithi122 2024-03-11T19:46:54.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:54 smithi107 ceph-mon[31712]: Deploying daemon osd.7 on smithi122 2024-03-11T19:46:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:55 smithi122 ceph-mon[36397]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:55 smithi145 ceph-mon[36730]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:55 smithi107 ceph-mon[31712]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:57 smithi122 ceph-mon[36397]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:57 smithi145 ceph-mon[36730]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:57 smithi107 ceph-mon[31712]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:59.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:59 smithi107 ceph-mon[31712]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:59.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:46:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:59 smithi122 ceph-mon[36397]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:46:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:59 smithi145 ceph-mon[36730]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:46:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:46:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:46:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:01 smithi122 ceph-mon[36397]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:01 smithi145 ceph-mon[36730]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:01 smithi107 ceph-mon[31712]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.921+0000 7f7b157fa700 1 -- 172.21.15.122:0/1451214598 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7b1802d8e0 con 0x7f7b000a1ac0 2024-03-11T19:47:01.926 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 7 on host 'smithi122' 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 msgr2=0x7f7b000a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 0x7f7b000a3f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7b18101ca0 tx=0x7f7b08009190 comp rx=0 tx=0).stop 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 msgr2=0x7f7b18109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c003790 tx=0x7f7b0c005b00 comp rx=0 tx=0).stop 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 shutdown_connections 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b000a1ac0 0x7f7b000a3f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.923+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b181027e0 0x7f7b1810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.924+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b18100c40 0x7f7b1810b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.924+0000 7f7b1e86e700 1 --2- 172.21.15.122:0/1451214598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b18101e40 0x7f7b18109ba0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.924+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 >> 172.21.15.122:0/1451214598 conn(0x7f7b180fc200 msgr2=0x7f7b181051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.924+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 shutdown_connections 2024-03-11T19:47:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:47:01.924+0000 7f7b1e86e700 1 -- 172.21.15.122:0/1451214598 wait complete. 2024-03-11T19:47:02.573 DEBUG:teuthology.orchestra.run.smithi122:osd.7> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.7.service 2024-03-11T19:47:02.576 INFO:tasks.cephadm:Deploying osd.8 on smithi145 with /dev/vg_nvme/lv_2... 2024-03-11T19:47:02.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:47:02.823 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:47:03.174 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.175 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.262 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:03.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:47:03.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:03.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-mon[36397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:47:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-mon[36397]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-mon[36397]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:47:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:04.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-mon[36397]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.014 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T19:47:03.936+0000 7f1ded250700 -1 osd.7 0 waiting for initial osdmap 2024-03-11T19:47:04.014 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 19:47:03 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T19:47:03.946+0000 7f1de9043700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:47:04.250 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:03 smithi145 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:47:04.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:03 smithi145 ceph-mon[36730]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:03 smithi145 ceph-mon[36730]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:47:04.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:04.251 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:03 smithi145 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:03 smithi107 ceph-mon[31712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:47:04.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:03 smithi107 ceph-mon[31712]: from='osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:03 smithi107 ceph-mon[31712]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:47:04.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:04.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:03 smithi107 ceph-mon[31712]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:47:04.804 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:47:04.805 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:47:04.805 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-03-11T19:47:04.805 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-03-11T19:47:04.805 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114362 s, 917 MB/s 2024-03-11T19:47:04.805 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-03-11T19:47:05.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:04 smithi145 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:47:05.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:04 smithi145 ceph-mon[36730]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:47:05.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:05.177 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:04 smithi145 ceph-mon[36730]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:05.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:04 smithi122 ceph-mon[36397]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:47:05.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:04 smithi122 ceph-mon[36397]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:47:05.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:05.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:04 smithi122 ceph-mon[36397]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:04 smithi107 ceph-mon[31712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:47:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:04 smithi107 ceph-mon[31712]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:47:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:05.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:04 smithi107 ceph-mon[31712]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:47:05.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi145:vg_nvme/lv_2 2024-03-11T19:47:05.714 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] boot 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:05 smithi122 ceph-mon[36397]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:47:06.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:47:06.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:47:06.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] boot 2024-03-11T19:47:06.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:47:06.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:06.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:05 smithi145 ceph-mon[36730]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: osd.7 [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] boot 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:47:06.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:05 smithi107 ceph-mon[31712]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:47:07.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.278+0000 7f868e84e700 1 -- 172.21.15.145:0/2252673880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688101ca0 msgr2=0x7f86881020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.278+0000 7f868e84e700 1 --2- 172.21.15.145:0/2252673880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688101ca0 0x7f86881020c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8678009990 tx=0x7f8678009ca0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- 172.21.15.145:0/2252673880 shutdown_connections 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- 172.21.15.145:0/2252673880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 0x7f868810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- 172.21.15.145:0/2252673880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688101ca0 0x7f86881020c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- 172.21.15.145:0/2252673880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8688102ea0 0x7f8688103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- 172.21.15.145:0/2252673880 >> 172.21.15.145:0/2252673880 conn(0x7f86880fd280 msgr2=0x7f86880ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- 172.21.15.145:0/2252673880 shutdown_connections 2024-03-11T19:47:07.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- 172.21.15.145:0/2252673880 wait complete. 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 Processor -- start 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- start start 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8688101ca0 0x7f8688197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 0x7f868819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819f980 con 0x7f8688101ca0 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819fac0 con 0x7f8688103840 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.279+0000 7f868e84e700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868819fc50 con 0x7f8688102ea0 2024-03-11T19:47:07.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40704/0 (socket says 172.21.15.145:40704) 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 -- 172.21.15.145:0/1258590943 learned_addr learned my addr 172.21.15.145:0/1258590943 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f868cdeb700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 0x7f868819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 -- 172.21.15.145:0/1258590943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 msgr2=0x7f868819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 0x7f868819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 -- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8688101ca0 msgr2=0x7f8688197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8688101ca0 0x7f8688197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 -- 172.21.15.145:0/1258590943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780095f0 con 0x7f8688102ea0 2024-03-11T19:47:07.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86877fe700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b560 tx=0x7f867c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c011830 con 0x7f8688102ea0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f867c011e70 con 0x7f8688102ea0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00f2e0 con 0x7f8688102ea0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868819ff30 con 0x7f8688102ea0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.280+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86881a0370 con 0x7f8688102ea0 2024-03-11T19:47:07.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.281+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8688109180 con 0x7f8688102ea0 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.282+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f867c0105f0 con 0x7f8688102ea0 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.283+0000 7f86857fa700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 0x7f86700a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.283+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f867c0cc810 con 0x7f8688102ea0 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.283+0000 7f8687fff700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 0x7f86700a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.284+0000 7f8687fff700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 0x7f86700a3f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86780095c0 tx=0x7f8678009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:07.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.286+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f867c0964c0 con 0x7f8688102ea0 2024-03-11T19:47:07.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:07.489+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f868802da20 con 0x7f86700a1ac0 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi122 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 5693M 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.495 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.496 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:07 smithi145 ceph-mon[36730]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi122 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 5693M 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:07.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:07.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.514 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:07 smithi122 ceph-mon[36397]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi122 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 5693M 2024-03-11T19:47:07.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:07.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:07 smithi107 ceph-mon[31712]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:47:08.450 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:08 smithi145 ceph-mon[36730]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:47:08.450 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:08.450 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:08.450 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:08 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:08.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:08 smithi122 ceph-mon[36397]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:47:08.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:08.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:08.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:08 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:08.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:08 smithi107 ceph-mon[31712]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:47:08.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:08.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:08.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:08 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:09.651 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:09 smithi107 ceph-mon[31712]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:09.651 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:09 smithi107 ceph-mon[31712]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:09.651 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:09 smithi107 ceph-mon[31712]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:47:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:09 smithi122 ceph-mon[36397]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:09 smithi122 ceph-mon[36397]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:09 smithi122 ceph-mon[36397]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:47:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:09 smithi145 ceph-mon[36730]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:09 smithi145 ceph-mon[36730]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:09 smithi145 ceph-mon[36730]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:47:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:10 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/3083665889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:10 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:10 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]': finished 2024-03-11T19:47:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:10 smithi122 ceph-mon[36397]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:47:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:10 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/3083665889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:10 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:10 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]': finished 2024-03-11T19:47:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:10 smithi145 ceph-mon[36730]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:47:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:10 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/3083665889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:10 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]: dispatch 2024-03-11T19:47:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:10 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0816612-8f19-48fe-9290-bebea9bcb91d"}]': finished 2024-03-11T19:47:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:10 smithi107 ceph-mon[31712]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:47:10.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:11.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:11 smithi122 ceph-mon[36397]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:11 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/1248024082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:11 smithi145 ceph-mon[36730]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:11 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/1248024082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:11 smithi107 ceph-mon[31712]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/1248024082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:13 smithi122 ceph-mon[36397]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:13.782 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:13 smithi145 ceph-mon[36730]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:13 smithi107 ceph-mon[31712]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:15.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:15 smithi122 ceph-mon[36397]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:15 smithi145 ceph-mon[36730]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:15 smithi107 ceph-mon[31712]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:17 smithi122 ceph-mon[36397]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:17 smithi145 ceph-mon[36730]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:17 smithi107 ceph-mon[31712]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:18.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:47:18.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:18 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:18.649 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:18 smithi145 ceph-mon[36730]: Deploying daemon osd.8 on smithi145 2024-03-11T19:47:18.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:47:18.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:18 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:18.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:18 smithi122 ceph-mon[36397]: Deploying daemon osd.8 on smithi145 2024-03-11T19:47:18.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:47:18.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:18 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:18.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:18 smithi107 ceph-mon[31712]: Deploying daemon osd.8 on smithi145 2024-03-11T19:47:19.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:19 smithi145 ceph-mon[36730]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:19 smithi122 ceph-mon[36397]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:19 smithi107 ceph-mon[31712]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:21.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:21 smithi145 ceph-mon[36730]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:21 smithi122 ceph-mon[36397]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:21 smithi107 ceph-mon[31712]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:23 smithi122 ceph-mon[36397]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:23 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:23 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:23 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:23 smithi145 ceph-mon[36730]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:23 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:23 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:23 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:23 smithi107 ceph-mon[31712]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:23 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:23 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:23 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:25.648 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:25 smithi145 ceph-mon[36730]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:25 smithi122 ceph-mon[36397]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:25 smithi107 ceph-mon[31712]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:27.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.223+0000 7f86857fa700 1 -- 172.21.15.145:0/1258590943 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f868802da20 con 0x7f86700a1ac0 2024-03-11T19:47:27.228 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 8 on host 'smithi145' 2024-03-11T19:47:27.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 msgr2=0x7f86700a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:27.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 0x7f86700a3f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86780095c0 tx=0x7f8678009510 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 msgr2=0x7f8688198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b560 tx=0x7f867c00b870 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 shutdown_connections 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f86700a1ac0 0x7f86700a3f80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688103840 0x7f868819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8688101ca0 0x7f8688197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 --2- 172.21.15.145:0/1258590943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8688102ea0 0x7f8688198330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 >> 172.21.15.145:0/1258590943 conn(0x7f86880fd280 msgr2=0x7f86881062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 shutdown_connections 2024-03-11T19:47:27.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:47:27.226+0000 7f868e84e700 1 -- 172.21.15.145:0/1258590943 wait complete. 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:27 smithi122 ceph-mon[36397]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:27 smithi145 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:27.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:27.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:27.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:27 smithi107 ceph-mon[31712]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:47:27.919 DEBUG:teuthology.orchestra.run.smithi145:osd.8> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.8.service 2024-03-11T19:47:27.921 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2024-03-11T19:47:27.921 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:28.369 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:47:28.463 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:47:28.463 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.463 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:47:28.464 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:28.464 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.464 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.464 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.464 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:28.464 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T19:47:28.145+0000 7fd72f7dc700 -1 osd.8 0 waiting for initial osdmap 2024-03-11T19:47:28.464 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 19:47:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T19:47:28.154+0000 7fd72b5cf700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:28.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:29.430 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:29 smithi145 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:47:29.431 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:29 smithi145 ceph-mon[36730]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:47:29.431 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.431 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.431 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:29 smithi145 ceph-mon[36730]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:29.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:29 smithi107 ceph-mon[31712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:47:29.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:29 smithi107 ceph-mon[31712]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:47:29.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:29 smithi107 ceph-mon[31712]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:29.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:29 smithi122 ceph-mon[36397]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:47:29.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:29 smithi122 ceph-mon[36397]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:47:29.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:29.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:29 smithi122 ceph-mon[36397]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] boot 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:47:30.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] boot 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:47:30.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: osd.8 [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] boot 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:47:30.524 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:47:31.474 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:47:31.474 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:47:31.475 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-11T19:47:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-11T19:47:31.475 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0339811 s, 309 MB/s 2024-03-11T19:47:31.475 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-11T19:47:31.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:47:31.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:31.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:31.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:47:31.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:31.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:31.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:32.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-03-11T19:47:32.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:32 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi145 2024-03-11T19:47:32.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:32 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 5693M 2024-03-11T19:47:32.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:32 smithi107 ceph-mon[31712]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:47:32.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:32 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi145 2024-03-11T19:47:32.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:32 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 5693M 2024-03-11T19:47:32.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:32 smithi122 ceph-mon[36397]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:47:32.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:32 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi145 2024-03-11T19:47:32.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:32 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 5693M 2024-03-11T19:47:32.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:32 smithi145 ceph-mon[36730]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:47:32.819 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:47:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:33 smithi107 ceph-mon[31712]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:47:33.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:33 smithi107 ceph-mon[31712]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:33 smithi122 ceph-mon[36397]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:47:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:33 smithi122 ceph-mon[36397]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:33.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:33 smithi145 ceph-mon[36730]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:47:33.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:33 smithi145 ceph-mon[36730]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:34.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:34 smithi122 ceph-mon[36397]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:47:34.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:34 smithi145 ceph-mon[36730]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:47:34.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:34 smithi107 ceph-mon[31712]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:47:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 -- 172.21.15.107:0/2107021663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe470109b70 msgr2=0x7fe47010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 --2- 172.21.15.107:0/2107021663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe470109b70 0x7fe47010c000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b0a0 tx=0x7fe46800b3b0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 -- 172.21.15.107:0/2107021663 shutdown_connections 2024-03-11T19:47:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 --2- 172.21.15.107:0/2107021663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe470109b70 0x7fe47010c000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 --2- 172.21.15.107:0/2107021663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe470074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 --2- 172.21.15.107:0/2107021663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4700746d0 0x7fe470074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 -- 172.21.15.107:0/2107021663 >> 172.21.15.107:0/2107021663 conn(0x7fe4700fd280 msgr2=0x7fe4700ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 -- 172.21.15.107:0/2107021663 shutdown_connections 2024-03-11T19:47:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.101+0000 7fe476800700 1 -- 172.21.15.107:0/2107021663 wait complete. 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.102+0000 7fe476800700 1 Processor -- start 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 -- start start 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe47019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 0x7fe4701a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701a1af0 con 0x7fe4700746d0 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701a1c30 con 0x7fe470109b70 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe476800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701a1dc0 con 0x7fe470075ca0 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe46f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe46f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50226/0 (socket says 172.21.15.107:50226) 2024-03-11T19:47:35.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe46f400700 1 -- 172.21.15.107:0/545015175 learned_addr learned my addr 172.21.15.107:0/545015175 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe46ea00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe47019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.103+0000 7fe46fe00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 0x7fe4701a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 -- 172.21.15.107:0/545015175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 msgr2=0x7fe4701a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 0x7fe4701a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 -- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 msgr2=0x7fe470199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 -- 172.21.15.107:0/545015175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009520 con 0x7fe470075ca0 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46f400700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46fe00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 0x7fe4701a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:35.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe46ea00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe47019a4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.104+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00cb70 con 0x7fe470075ca0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.105+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe468009d00 con 0x7fe470075ca0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.105+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe45c00ccd0 con 0x7fe470075ca0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.105+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c018680 con 0x7fe470075ca0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.105+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4701a2410 con 0x7fe470075ca0 2024-03-11T19:47:35.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.106+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe470108930 con 0x7fe470075ca0 2024-03-11T19:47:35.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.108+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe45c010430 con 0x7fe470075ca0 2024-03-11T19:47:35.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.109+0000 7fe467e00700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 0x7fe4540a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:35.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.109+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fe45c014070 con 0x7fe470075ca0 2024-03-11T19:47:35.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.109+0000 7fe46f400700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 0x7fe4540a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:35.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.110+0000 7fe46f400700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 0x7fe4540a3f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe460005950 tx=0x7fe460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:35.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.112+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe45c096a20 con 0x7fe470075ca0 2024-03-11T19:47:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:47:35.347+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe47002d870 con 0x7fe4540a1ac0 2024-03-11T19:47:35.459 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:35 smithi145 ceph-mon[36730]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:35.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:35 smithi122 ceph-mon[36397]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:35.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:35 smithi107 ceph-mon[31712]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:36.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:36 smithi107 ceph-mon[31712]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:36.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:36.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:36.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:36.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:36 smithi122 ceph-mon[36397]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:36.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:36.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:36.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:36 smithi145 ceph-mon[36730]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:47:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:47:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:47:36.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:37.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:37 smithi122 ceph-mon[36397]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:47:37.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:37 smithi145 ceph-mon[36730]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:47:37.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:37 smithi107 ceph-mon[31712]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:47:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:39 smithi122 ceph-mon[36397]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:47:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:39 smithi145 ceph-mon[36730]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:47:39.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:39 smithi107 ceph-mon[31712]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-11T19:47:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:40 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]: dispatch 2024-03-11T19:47:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:40 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]': finished 2024-03-11T19:47:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:40 smithi107 ceph-mon[31712]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:47:40.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:47:40.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:40 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]: dispatch 2024-03-11T19:47:40.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:40 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]': finished 2024-03-11T19:47:40.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:40 smithi122 ceph-mon[36397]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:47:40.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:47:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:40 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]: dispatch 2024-03-11T19:47:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:40 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/802951800' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8774d5-15de-4134-82a4-fdeb3ff19f2a"}]': finished 2024-03-11T19:47:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:40 smithi145 ceph-mon[36730]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:47:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:47:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:41 smithi122 ceph-mon[36397]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:47:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:41 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1700004197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:41 smithi145 ceph-mon[36730]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:47:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:41 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1700004197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:41.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:41 smithi107 ceph-mon[31712]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:47:41.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:41 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1700004197' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:47:43.709 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:43 smithi107 ceph-mon[31712]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:47:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:43 smithi122 ceph-mon[36397]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:47:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:43 smithi145 ceph-mon[36730]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-11T19:47:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:45 smithi122 ceph-mon[36397]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:47:45.775 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:45 smithi107 ceph-mon[31712]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:47:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:45 smithi145 ceph-mon[36730]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:47:47.606 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:47 smithi107 ceph-mon[31712]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:47.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:47 smithi122 ceph-mon[36397]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:47 smithi145 ceph-mon[36730]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:49 smithi107 ceph-mon[31712]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:47:49.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:49 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:49 smithi122 ceph-mon[36397]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:47:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:49 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:49 smithi145 ceph-mon[36730]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:47:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:49 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:50.519 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:50 smithi107 ceph-mon[31712]: Deploying daemon osd.9 on smithi107 2024-03-11T19:47:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:50 smithi122 ceph-mon[36397]: Deploying daemon osd.9 on smithi107 2024-03-11T19:47:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:50 smithi145 ceph-mon[36730]: Deploying daemon osd.9 on smithi107 2024-03-11T19:47:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:51 smithi122 ceph-mon[36397]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:51 smithi145 ceph-mon[36730]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:51.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:51 smithi107 ceph-mon[31712]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:53 smithi122 ceph-mon[36397]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:53 smithi145 ceph-mon[36730]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:53 smithi107 ceph-mon[31712]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:55.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:55 smithi107 ceph-mon[31712]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:55 smithi122 ceph-mon[36397]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:55 smithi145 ceph-mon[36730]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:57.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:57 smithi107 ceph-mon[31712]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:57 smithi122 ceph-mon[36397]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:57 smithi145 ceph-mon[36730]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:59.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:58 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:58 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:58 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:59.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:47:58 smithi107 ceph-mon[31712]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:58 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:58 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:58 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:47:58 smithi122 ceph-mon[36397]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:47:59.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:58 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:58 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:47:59.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:58 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:59.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:47:58 smithi145 ceph-mon[36730]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:01 smithi122 ceph-mon[36397]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:01 smithi145 ceph-mon[36730]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:01 smithi107 ceph-mon[31712]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:02.541 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:02 smithi107 ceph-mon[31712]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.541 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:02 smithi107 ceph-mon[31712]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.541 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:02.541 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:02 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:02 smithi122 ceph-mon[36397]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:02 smithi122 ceph-mon[36397]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:02 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:02 smithi145 ceph-mon[36730]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:02 smithi145 ceph-mon[36730]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:48:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:02 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.093+0000 7fe467e00700 1 -- 172.21.15.107:0/545015175 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe47002d870 con 0x7fe4540a1ac0 2024-03-11T19:48:03.100 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 msgr2=0x7fe4540a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 0x7fe4540a3f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe460005950 tx=0x7fe460009260 comp rx=0 tx=0).stop 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 msgr2=0x7fe47019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe47019a4a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00ed00 comp rx=0 tx=0).stop 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 shutdown_connections 2024-03-11T19:48:03.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4540a1ac0 0x7fe4540a3f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe470109b70 0x7fe4701a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4700746d0 0x7fe470199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 --2- 172.21.15.107:0/545015175 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe470075ca0 0x7fe47019a4a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.098+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 >> 172.21.15.107:0/545015175 conn(0x7fe4700fd280 msgr2=0x7fe470105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.099+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 shutdown_connections 2024-03-11T19:48:03.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:03.099+0000 7fe476800700 1 -- 172.21.15.107:0/545015175 wait complete. 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:03 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:48:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:48:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:03.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:03 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:03 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:03.950 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.9.service 2024-03-11T19:48:03.955 INFO:tasks.cephadm:Deploying osd.10 on smithi122 with /dev/vg_nvme/lv_1... 2024-03-11T19:48:03.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:48:04.198 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:48:04.649 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:04 smithi122 ceph-mon[36397]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:48:04.649 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:04 smithi122 ceph-mon[36397]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:48:04.649 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:04.649 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:04 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:04.766 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:04 smithi107 ceph-mon[31712]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:48:04.766 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:04 smithi107 ceph-mon[31712]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:48:04.766 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:04.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:04 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:04 smithi145 ceph-mon[36730]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-11T19:48:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:04 smithi145 ceph-mon[36730]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:48:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:04.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:04 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:05.613 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:48:05.613 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:48:05.614 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:05.614 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] boot 2024-03-11T19:48:05.614 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:48:05.614 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:05 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:05.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:48:05.688 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:48:05.689 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:05.689 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] boot 2024-03-11T19:48:05.689 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:48:05.689 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:05 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:48:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:48:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:48:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: osd.9 [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] boot 2024-03-11T19:48:05.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:48:05.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:05 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113585 s, 923 MB/s 2024-03-11T19:48:06.274 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-03-11T19:48:06.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:06 smithi107 ceph-mon[31712]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:48:06.575 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:06 smithi122 ceph-mon[36397]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:48:06.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:06 smithi145 ceph-mon[36730]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:48:06.994 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-03-11T19:48:07.234 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:48:07.668 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:07 smithi122 ceph-mon[36397]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:07 smithi145 ceph-mon[36730]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:07.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:07 smithi107 ceph-mon[31712]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 -- 172.21.15.122:0/358886561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 msgr2=0x7f7ecc1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 --2- 172.21.15.122:0/358886561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc1020c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0099c0 tx=0x7f7ebc009cd0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 -- 172.21.15.122:0/358886561 shutdown_connections 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 --2- 172.21.15.122:0/358886561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ecc103840 0x7f7ecc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 --2- 172.21.15.122:0/358886561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc1020c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 --2- 172.21.15.122:0/358886561 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 0x7f7ecc103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 -- 172.21.15.122:0/358886561 >> 172.21.15.122:0/358886561 conn(0x7f7ecc0fd280 msgr2=0x7f7ecc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 -- 172.21.15.122:0/358886561 shutdown_connections 2024-03-11T19:48:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.798+0000 7f7ed2280700 1 -- 172.21.15.122:0/358886561 wait complete. 2024-03-11T19:48:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 Processor -- start 2024-03-11T19:48:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 -- start start 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 0x7f7ecc074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ecc103840 0x7f7ecc0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc19bb30 con 0x7f7ecc103840 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc074ae0 con 0x7f7ecc101ca0 2024-03-11T19:48:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ed2280700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc074ca0 con 0x7f7ecc102ea0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58678/0 (socket says 172.21.15.122:58678) 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecbfff700 1 -- 172.21.15.122:0/2147095436 learned_addr learned my addr 172.21.15.122:0/2147095436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecb7fe700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 0x7f7ecc074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecbfff700 1 -- 172.21.15.122:0/2147095436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 msgr2=0x7f7ecc074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.799+0000 7f7ecbfff700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 0x7f7ecc074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ecbfff700 1 -- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ecc103840 msgr2=0x7f7ecc0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ecbfff700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ecc103840 0x7f7ecc0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ecbfff700 1 -- 172.21.15.122:0/2147095436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0095f0 con 0x7f7ecc101ca0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ecbfff700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0038c0 tx=0x7f7ebc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc01d070 con 0x7f7ecc101ca0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc1a6500 con 0x7f7ecc101ca0 2024-03-11T19:48:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc004110 con 0x7f7ecc101ca0 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1a68b0 con 0x7f7ecc101ca0 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.800+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc004a20 con 0x7f7ecc101ca0 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.801+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc1090e0 con 0x7f7ecc101ca0 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.802+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7ebc005150 con 0x7f7ecc101ca0 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.802+0000 7f7ec97fa700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 0x7f7eb40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.803+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7ebc0cd9d0 con 0x7f7ecc101ca0 2024-03-11T19:48:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.803+0000 7f7ecb7fe700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 0x7f7eb40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.804+0000 7f7ecb7fe700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 0x7f7eb40a3f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005950 tx=0x7f7ec000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:08.807+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ebc05cd20 con 0x7f7ecc101ca0 2024-03-11T19:48:09.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:09.007+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ecc0620b0 con 0x7f7eb40a1ac0 2024-03-11T19:48:09.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:09.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.759 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.760 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi107 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.651 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.652 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:10.652 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.652 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.652 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:10.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi107 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi107 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:11 smithi122 ceph-mon[36397]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:11 smithi145 ceph-mon[36730]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:11 smithi107 ceph-mon[31712]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:12 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/1141131757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:12 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:12 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]': finished 2024-03-11T19:48:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:12 smithi122 ceph-mon[36397]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:48:12.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:12 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:12.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:12 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/1141131757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:12 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:12 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]': finished 2024-03-11T19:48:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:12 smithi145 ceph-mon[36730]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:48:12.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:12 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:12 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/1141131757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:12 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]: dispatch 2024-03-11T19:48:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:12 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd"}]': finished 2024-03-11T19:48:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:12 smithi107 ceph-mon[31712]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:48:12.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:12 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:13.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:13 smithi122 ceph-mon[36397]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:13 smithi122 ceph-mon[36397]: from='client.? 172.21.15.122:0/447037420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:13 smithi145 ceph-mon[36730]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:13 smithi145 ceph-mon[36730]: from='client.? 172.21.15.122:0/447037420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:13 smithi107 ceph-mon[31712]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:13 smithi107 ceph-mon[31712]: from='client.? 172.21.15.122:0/447037420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:15 smithi122 ceph-mon[36397]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:15 smithi145 ceph-mon[36730]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:15 smithi107 ceph-mon[31712]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:17.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:17 smithi122 ceph-mon[36397]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:17 smithi145 ceph-mon[36730]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:17 smithi107 ceph-mon[31712]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:19.607 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:19 smithi122 ceph-mon[36397]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:19.607 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:19 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:48:19.607 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:19 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:19 smithi145 ceph-mon[36730]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:19 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:48:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:19 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:19 smithi107 ceph-mon[31712]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:19 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:48:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:19 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:20.562 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:20 smithi122 ceph-mon[36397]: Deploying daemon osd.10 on smithi122 2024-03-11T19:48:20.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:20 smithi145 ceph-mon[36730]: Deploying daemon osd.10 on smithi122 2024-03-11T19:48:20.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:20 smithi107 ceph-mon[31712]: Deploying daemon osd.10 on smithi122 2024-03-11T19:48:21.612 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:21 smithi122 ceph-mon[36397]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:21 smithi145 ceph-mon[36730]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:21 smithi107 ceph-mon[31712]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:23.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:23 smithi122 ceph-mon[36397]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:23 smithi145 ceph-mon[36730]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:23 smithi107 ceph-mon[31712]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:25.740 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:25 smithi122 ceph-mon[36397]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:25.741 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:25 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.741 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:25 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.741 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:25 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:25 smithi145 ceph-mon[36730]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:25 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:25 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:25 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:25 smithi107 ceph-mon[31712]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:25.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:27.476 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:27 smithi122 ceph-mon[36397]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:27 smithi145 ceph-mon[36730]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:27 smithi107 ceph-mon[31712]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:28.498 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 10 on host 'smithi122' 2024-03-11T19:48:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.493+0000 7f7ec97fa700 1 -- 172.21.15.122:0/2147095436 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7ecc0620b0 con 0x7f7eb40a1ac0 2024-03-11T19:48:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 msgr2=0x7f7eb40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 0x7f7eb40a3f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005950 tx=0x7f7ec000b410 comp rx=0 tx=0).stop 2024-03-11T19:48:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 msgr2=0x7f7ecc075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0038c0 tx=0x7f7ebc004730 comp rx=0 tx=0).stop 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 shutdown_connections 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7eb40a1ac0 0x7f7eb40a3f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ecc103840 0x7f7ecc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc101ca0 0x7f7ecc075a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 --2- 172.21.15.122:0/2147095436 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7ecc102ea0 0x7f7ecc074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.495+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 >> 172.21.15.122:0/2147095436 conn(0x7f7ecc0fd280 msgr2=0x7f7ecc106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.496+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 shutdown_connections 2024-03-11T19:48:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-11T19:48:28.496+0000 7f7ed2280700 1 -- 172.21.15.122:0/2147095436 wait complete. 2024-03-11T19:48:29.164 DEBUG:teuthology.orchestra.run.smithi122:osd.10> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.10.service 2024-03-11T19:48:29.166 INFO:tasks.cephadm:Deploying osd.11 on smithi145 with /dev/vg_nvme/lv_1... 2024-03-11T19:48:29.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:48:29.415 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:29.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:29 smithi107 ceph-mon[31712]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:29.749 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:29 smithi122 ceph-mon[36397]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:29.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:29.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.764 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:29.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:29.765 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:29 smithi145 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:48:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:48:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:48:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:30 smithi122 ceph-mon[36397]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:31.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:48:31.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:48:31.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:31.073 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.074 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:30 smithi145 ceph-mon[36730]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:31.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:48:31.294 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: from='osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:48:31.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:31.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-03-11T19:48:31.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:30 smithi107 ceph-mon[31712]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr: stderr: 10+0 records in 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr:10+0 records out 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128145 s, 818 MB/s 2024-03-11T19:48:31.536 INFO:teuthology.orchestra.run.smithi145.stderr:--> Zapping successful for: 2024-03-11T19:48:31.558 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T19:48:31.272+0000 7fdb24980700 -1 osd.10 0 waiting for initial osdmap 2024-03-11T19:48:31.558 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T19:48:31.285+0000 7fdb1c76b700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:48:32.246 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-mon[36397]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:48:32.246 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-mon[36397]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:48:32.246 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.246 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-mon[36397]: from='osd.10 ' entity='osd.10' 2024-03-11T19:48:32.247 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:31 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:31 smithi145 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:48:32.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:31 smithi145 ceph-mon[36730]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:48:32.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:31 smithi145 ceph-mon[36730]: from='osd.10 ' entity='osd.10' 2024-03-11T19:48:32.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:31 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:31 smithi107 ceph-mon[31712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-03-11T19:48:32.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:31 smithi107 ceph-mon[31712]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:48:32.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:31 smithi107 ceph-mon[31712]: from='osd.10 ' entity='osd.10' 2024-03-11T19:48:32.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:31 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:32.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi145:vg_nvme/lv_1 2024-03-11T19:48:32.590 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] boot 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi122 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 4270M 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.039 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] boot 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi122 2024-03-11T19:48:33.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 4270M 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: osd.10 [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] boot 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi122 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 4270M 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:33.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- 172.21.15.145:0/3370300795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 msgr2=0x7f9650100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 --2- 172.21.15.145:0/3370300795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650100ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9640009990 tx=0x7f9640009ca0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- 172.21.15.145:0/3370300795 shutdown_connections 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 --2- 172.21.15.145:0/3370300795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650102770 0x7f9650109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 --2- 172.21.15.145:0/3370300795 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650100ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 --2- 172.21.15.145:0/3370300795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9650101dd0 0x7f9650102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- 172.21.15.145:0/3370300795 >> 172.21.15.145:0/3370300795 conn(0x7f96500fc190 msgr2=0x7f96500fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- 172.21.15.145:0/3370300795 shutdown_connections 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- 172.21.15.145:0/3370300795 wait complete. 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 Processor -- start 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.227+0000 7f965617b700 1 -- start start 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 0x7f965019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9650102770 0x7f96501a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501a1a70 con 0x7f9650102770 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501a1bb0 con 0x7f9650101dd0 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501a1d40 con 0x7f9650100bd0 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51436/0 (socket says 172.21.15.145:51436) 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 -- 172.21.15.145:0/3888909014 learned_addr learned my addr 172.21.15.145:0/3888909014 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:48:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f9646ffd700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 0x7f965019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 -- 172.21.15.145:0/3888909014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 msgr2=0x7f965019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 0x7f965019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 -- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9650102770 msgr2=0x7f96501a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9650102770 0x7f96501a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 -- 172.21.15.145:0/3888909014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400095f0 con 0x7f9650100bd0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f9646ffd700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 0x7f965019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964f7fe700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9640003760 tx=0x7f964000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964001d070 con 0x7f9650100bd0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f964000fce0 con 0x7f9650100bd0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501a1fc0 con 0x7f9650100bd0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640017770 con 0x7f9650100bd0 2024-03-11T19:48:34.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.228+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501a23f0 con 0x7f9650100bd0 2024-03-11T19:48:34.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.229+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650068930 con 0x7f9650100bd0 2024-03-11T19:48:34.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.230+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9640022a50 con 0x7f9650100bd0 2024-03-11T19:48:34.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.231+0000 7f964d7fa700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 0x7f963c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:34.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.231+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f96400ce270 con 0x7f9650100bd0 2024-03-11T19:48:34.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.231+0000 7f9646ffd700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 0x7f963c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:34.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.232+0000 7f9646ffd700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 0x7f963c0a3f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96380099d0 tx=0x7f9638008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:34.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.235+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f964005dd20 con 0x7f9650100bd0 2024-03-11T19:48:34.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:34.442+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f96500fe350 con 0x7f963c0a1ac0 2024-03-11T19:48:34.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:34 smithi122 ceph-mon[36397]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:48:34.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:34 smithi145 ceph-mon[36730]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:48:34.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:34 smithi107 ceph-mon[31712]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:35.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:35.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:48:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi145:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:48:35.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:48:35.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:48:35.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:36 smithi122 ceph-mon[36397]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:48:36.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:36 smithi145 ceph-mon[36730]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:48:36.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:36 smithi107 ceph-mon[31712]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/3518816390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]': finished 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:48:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:37 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:48:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/3518816390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]': finished 2024-03-11T19:48:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:48:37.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:37 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:48:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/3518816390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]: dispatch 2024-03-11T19:48:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "007f1865-b782-4f5e-98f1-b358a65840f8"}]': finished 2024-03-11T19:48:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:48:37.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:37 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:38.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:38 smithi122 ceph-mon[36397]: from='client.? 172.21.15.145:0/2540342714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:38 smithi145 ceph-mon[36730]: from='client.? 172.21.15.145:0/2540342714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:38.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:38 smithi107 ceph-mon[31712]: from='client.? 172.21.15.145:0/2540342714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:48:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:39 smithi122 ceph-mon[36397]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:48:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:39 smithi145 ceph-mon[36730]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:48:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:39 smithi107 ceph-mon[31712]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-11T19:48:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:41 smithi122 ceph-mon[36397]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:48:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:41 smithi145 ceph-mon[36730]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:48:41.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:41 smithi107 ceph-mon[31712]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:48:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:43 smithi122 ceph-mon[36397]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-11T19:48:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:43 smithi145 ceph-mon[36730]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-11T19:48:43.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:43 smithi107 ceph-mon[31712]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-11T19:48:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:45 smithi122 ceph-mon[36397]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:45.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:45 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:48:45.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:45 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:45.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:45 smithi145 ceph-mon[36730]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:45.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:45 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:48:45.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:45 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:45.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:45 smithi107 ceph-mon[31712]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:45.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:45 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:48:45.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:45 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:46.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:46 smithi122 ceph-mon[36397]: Deploying daemon osd.11 on smithi145 2024-03-11T19:48:46.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:46 smithi145 ceph-mon[36730]: Deploying daemon osd.11 on smithi145 2024-03-11T19:48:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:46 smithi107 ceph-mon[31712]: Deploying daemon osd.11 on smithi145 2024-03-11T19:48:47.627 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:47 smithi145 ceph-mon[36730]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:47.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:47 smithi122 ceph-mon[36397]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:47.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:47 smithi107 ceph-mon[31712]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:49 smithi122 ceph-mon[36397]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:49 smithi145 ceph-mon[36730]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:49 smithi107 ceph-mon[31712]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:51.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:51.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:51 smithi122 ceph-mon[36397]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:51.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:51.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:51 smithi145 ceph-mon[36730]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:51.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:51.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:51.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:51 smithi107 ceph-mon[31712]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:53.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:53.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:53.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:53.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:53.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:53.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:53.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.975+0000 7f964d7fa700 1 -- 172.21.15.145:0/3888909014 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f96500fe350 con 0x7f963c0a1ac0 2024-03-11T19:48:53.980 INFO:teuthology.orchestra.run.smithi145.stdout:Created osd(s) 11 on host 'smithi145' 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 msgr2=0x7f963c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 0x7f963c0a3f80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96380099d0 tx=0x7f9638008040 comp rx=0 tx=0).stop 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 msgr2=0x7f9650199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9640003760 tx=0x7f964000f740 comp rx=0 tx=0).stop 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 shutdown_connections 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f963c0a1ac0 0x7f963c0a3f80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:53.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9650102770 0x7f96501a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9650100bd0 0x7f9650199f50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 --2- 172.21.15.145:0/3888909014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9650101dd0 0x7f965019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 >> 172.21.15.145:0/3888909014 conn(0x7f96500fc190 msgr2=0x7f96500fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 shutdown_connections 2024-03-11T19:48:53.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:48:53.977+0000 7f965617b700 1 -- 172.21.15.145:0/3888909014 wait complete. 2024-03-11T19:48:54.582 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:54.582 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.582 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.583 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:54 smithi145 ceph-mon[36730]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:54.583 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:54 smithi145 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:54.608 DEBUG:teuthology.orchestra.run.smithi145:osd.11> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.11.service 2024-03-11T19:48:54.610 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-11T19:48:54.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-11T19:48:54.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:54 smithi107 ceph-mon[31712]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:54 smithi107 ceph-mon[31712]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:54 smithi122 ceph-mon[36397]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:54 smithi122 ceph-mon[36397]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:48:55.056 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:48:55.516 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:55.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:48:55.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:55.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:48:55.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:55.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:55 smithi107 ceph-mon[31712]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:55.673 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: from='osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:55 smithi122 ceph-mon[36397]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]: dispatch 2024-03-11T19:48:56.287 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T19:48:55.943+0000 7fb6d436a700 -1 osd.11 0 waiting for initial osdmap 2024-03-11T19:48:56.288 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 19:48:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T19:48:55.955+0000 7fb6d015d700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:48:57.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:56 smithi145 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:48:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:56 smithi145 ceph-mon[36730]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:48:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:56 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:57.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:56 smithi145 ceph-mon[36730]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:57.134 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:56 smithi107 ceph-mon[31712]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:48:57.135 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:56 smithi107 ceph-mon[31712]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:48:57.135 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:56 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:57.135 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:56 smithi107 ceph-mon[31712]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:57.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:56 smithi122 ceph-mon[36397]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi145", "root=default"]}]': finished 2024-03-11T19:48:57.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:56 smithi122 ceph-mon[36397]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:48:57.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:56 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:57.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:56 smithi122 ceph-mon[36397]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 -- 172.21.15.107:0/2578638241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48101be0 msgr2=0x7f1f48102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/2578638241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48101be0 0x7f1f48102000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009920 tx=0x7f1f3c009c30 comp rx=0 tx=0).stop 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 -- 172.21.15.107:0/2578638241 shutdown_connections 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/2578638241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 0x7f1f4810a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/2578638241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48101be0 0x7f1f48102000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/2578638241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48102de0 0x7f1f48103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 -- 172.21.15.107:0/2578638241 >> 172.21.15.107:0/2578638241 conn(0x7f1f480fd180 msgr2=0x7f1f480ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 -- 172.21.15.107:0/2578638241 shutdown_connections 2024-03-11T19:48:57.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.477+0000 7f1f4ec00700 1 -- 172.21.15.107:0/2578638241 wait complete. 2024-03-11T19:48:57.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.478+0000 7f1f4ec00700 1 Processor -- start 2024-03-11T19:48:57.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.478+0000 7f1f4ec00700 1 -- start start 2024-03-11T19:48:57.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.478+0000 7f1f4ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.478+0000 7f1f4ec00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48102de0 0x7f1f48074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4ec00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 0x7f1f480745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48074d40 con 0x7f1f48101be0 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4ec00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48074eb0 con 0x7f1f48102de0 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4ec00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481a1c60 con 0x7f1f48103780 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55010/0 (socket says 172.21.15.107:55010) 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 -- 172.21.15.107:0/1917310887 learned_addr learned my addr 172.21.15.107:0/1917310887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4ce00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48102de0 0x7f1f48074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4e200700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 0x7f1f480745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 -- 172.21.15.107:0/1917310887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 msgr2=0x7f1f480745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 0x7f1f480745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 -- 172.21.15.107:0/1917310887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48102de0 msgr2=0x7f1f48074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48102de0 0x7f1f48074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.479+0000 7f1f4d800700 1 -- 172.21.15.107:0/1917310887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c009520 con 0x7f1f48101be0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f4d800700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009230 tx=0x7f1f3c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01c070 con 0x7f1f48101be0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f481a1e90 con 0x7f1f48101be0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c00f910 con 0x7f1f48101be0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003a70 con 0x7f1f48101be0 2024-03-11T19:48:57.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.480+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f481a2240 con 0x7f1f48101be0 2024-03-11T19:48:57.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.481+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f481090c0 con 0x7f1f48101be0 2024-03-11T19:48:57.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.484+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1f3c020030 con 0x7f1f48101be0 2024-03-11T19:48:57.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.485+0000 7f1f46000700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 0x7f1f300a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.485+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f3c0cdf90 con 0x7f1f48101be0 2024-03-11T19:48:57.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.485+0000 7f1f4ce00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 0x7f1f300a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.486+0000 7f1f4ce00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 0x7f1f300a4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009c60 tx=0x7f1f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.488+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f3c097630 con 0x7f1f48101be0 2024-03-11T19:48:57.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.721+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1f4810d520 con 0x7f1f48101be0 2024-03-11T19:48:57.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.722+0000 7f1f46000700 1 -- 172.21.15.107:0/1917310887 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1f3c09acf0 con 0x7f1f48101be0 2024-03-11T19:48:57.727 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:48:57.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.726+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 msgr2=0x7f1f300a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.726+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 0x7f1f300a4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009c60 tx=0x7f1f34009260 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.726+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 msgr2=0x7f1f480759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.726+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009230 tx=0x7f1f3c005250 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 shutdown_connections 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f300a1b40 0x7f1f300a4000 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009c60 tx=0x7f1f34009260 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103780 0x7f1f480745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48101be0 0x7f1f480759c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 --2- 172.21.15.107:0/1917310887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f48102de0 0x7f1f48074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 >> 172.21.15.107:0/1917310887 conn(0x7f1f480fd180 msgr2=0x7f1f48106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 shutdown_connections 2024-03-11T19:48:57.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:48:57.727+0000 7f1f4ec00700 1 -- 172.21.15.107:0/1917310887 wait complete. 2024-03-11T19:48:58.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: purged_snaps scrub starts 2024-03-11T19:48:58.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: purged_snaps scrub ok 2024-03-11T19:48:58.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] boot 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: Detected new or changed devices on smithi145 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:58.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:57 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1917310887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: purged_snaps scrub starts 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: purged_snaps scrub ok 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] boot 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: Detected new or changed devices on smithi145 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.264 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:57 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1917310887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:48:58.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: purged_snaps scrub starts 2024-03-11T19:48:58.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: purged_snaps scrub ok 2024-03-11T19:48:58.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: osd.11 [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] boot 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: Detected new or changed devices on smithi145 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:48:58.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:57 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1917310887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:48:58.618 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710186536,"num_in_osds":12,"osd_in_since":1710186517,"num_remapped_pgs":0} 2024-03-11T19:48:58.619 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:48:59.042 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:48:59.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:58 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 4270M 2024-03-11T19:48:59.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:58 smithi107 ceph-mon[31712]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:48:59.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:48:58 smithi107 ceph-mon[31712]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:48:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:58 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 4270M 2024-03-11T19:48:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:58 smithi122 ceph-mon[36397]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:48:59.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:48:58 smithi122 ceph-mon[36397]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:48:59.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:58 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 4270M 2024-03-11T19:48:59.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:58 smithi145 ceph-mon[36730]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:48:59.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:48:58 smithi145 ceph-mon[36730]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:49:01.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 -- 172.21.15.107:0/1674187088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41091a0 msgr2=0x7f77a4110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:01.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 --2- 172.21.15.107:0/1674187088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41091a0 0x7f77a4110d70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f779c00b0a0 tx=0x7f779c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 -- 172.21.15.107:0/1674187088 shutdown_connections 2024-03-11T19:49:01.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 --2- 172.21.15.107:0/1674187088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41091a0 0x7f77a4110d70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 --2- 172.21.15.107:0/1674187088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a41040c0 0x7f77a41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 --2- 172.21.15.107:0/1674187088 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a4104a70 0x7f77a4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 -- 172.21.15.107:0/1674187088 >> 172.21.15.107:0/1674187088 conn(0x7f77a40ff970 msgr2=0x7f77a4101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 -- 172.21.15.107:0/1674187088 shutdown_connections 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.366+0000 7f77abc00700 1 -- 172.21.15.107:0/1674187088 wait complete. 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.367+0000 7f77abc00700 1 Processor -- start 2024-03-11T19:49:01.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 -- start start 2024-03-11T19:49:01.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 0x7f77a419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a41091a0 0x7f77a419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a419f460 con 0x7f77a41040c0 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a419f5d0 con 0x7f77a4104a70 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77abc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a419f790 con 0x7f77a41091a0 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77a9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77a9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55016/0 (socket says 172.21.15.107:55016) 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77a9200700 1 -- 172.21.15.107:0/1142018503 learned_addr learned my addr 172.21.15.107:0/1142018503 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.368+0000 7f77a3e00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 0x7f77a419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a9c00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a41091a0 0x7f77a419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 -- 172.21.15.107:0/1142018503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a41091a0 msgr2=0x7f77a419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a41091a0 0x7f77a419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 -- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 msgr2=0x7f77a419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 -- 172.21.15.107:0/1142018503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c009d00 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a9200700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a3e00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 0x7f77a419aac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9c0 tx=0x7f779000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.369+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000c9c0 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.370+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7790010430 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.370+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790018640 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.370+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a419fa70 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.370+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a41a72d0 con 0x7f77a4104a70 2024-03-11T19:49:01.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.371+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a410c4c0 con 0x7f77a4104a70 2024-03-11T19:49:01.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.373+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f77900105a0 con 0x7f77a4104a70 2024-03-11T19:49:01.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.374+0000 7f77a1600700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 0x7f778c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:01.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.374+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7790014070 con 0x7f77a4104a70 2024-03-11T19:49:01.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.374+0000 7f77a9200700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 0x7f778c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:01.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.375+0000 7f77a9200700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 0x7f778c0a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7798011550 tx=0x7f7798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:01.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.377+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7790096a90 con 0x7f77a4104a70 2024-03-11T19:49:01.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:01 smithi107 ceph-mon[31712]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:01.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.601+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a419b8e0 con 0x7f77a4104a70 2024-03-11T19:49:01.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.602+0000 7f77a1600700 1 -- 172.21.15.107:0/1142018503 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18548 (secure 0 0 0) 0x7f779009a150 con 0x7f77a4104a70 2024-03-11T19:49:01.608 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:01.608 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:48:58.522066+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 msgr2=0x7f778c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 0x7f778c0a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7798011550 tx=0x7f7798009260 comp rx=0 tx=0).stop 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 msgr2=0x7f77a419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 0x7f77a419aac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9c0 tx=0x7f779000ecd0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 shutdown_connections 2024-03-11T19:49:01.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f778c0a1b40 0x7f778c0a4000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77a41091a0 0x7f77a419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a41040c0 0x7f77a419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 --2- 172.21.15.107:0/1142018503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77a4104a70 0x7f77a419aac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 >> 172.21.15.107:0/1142018503 conn(0x7f77a40ff970 msgr2=0x7f77a41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 shutdown_connections 2024-03-11T19:49:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:01.605+0000 7f77abc00700 1 -- 172.21.15.107:0/1142018503 wait complete. 2024-03-11T19:49:01.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:01 smithi122 ceph-mon[36397]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:01 smithi145 ceph-mon[36730]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:02.521 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:49:02.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-11T19:49:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1142018503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1142018503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1142018503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:02.965 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:49:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:03 smithi122 ceph-mon[36397]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:03 smithi145 ceph-mon[36730]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:03 smithi107 ceph-mon[31712]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:05.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 -- 172.21.15.107:0/1863576867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8104670 msgr2=0x7fd6c8104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:05.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/1863576867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8104670 0x7fd6c8104b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009f90 tx=0x7fd6c000b330 comp rx=0 tx=0).stop 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 -- 172.21.15.107:0/1863576867 shutdown_connections 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/1863576867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8104670 0x7fd6c8104b00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/1863576867 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8106b60 0x7fd6c8069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/1863576867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8069e60 0x7fd6c806a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 -- 172.21.15.107:0/1863576867 >> 172.21.15.107:0/1863576867 conn(0x7fd6c8077aa0 msgr2=0x7fd6c8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 -- 172.21.15.107:0/1863576867 shutdown_connections 2024-03-11T19:49:05.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.252+0000 7fd6cdc00700 1 -- 172.21.15.107:0/1863576867 wait complete. 2024-03-11T19:49:05.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.253+0000 7fd6cdc00700 1 Processor -- start 2024-03-11T19:49:05.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 -- start start 2024-03-11T19:49:05.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:05.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8104670 0x7fd6c81a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:05.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8106b60 0x7fd6c819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c819eef0 con 0x7fd6c8069e60 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c819f060 con 0x7fd6c8106b60 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.254+0000 7fd6cdc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c819f220 con 0x7fd6c8104670 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48196/0 (socket says 172.21.15.107:48196) 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 -- 172.21.15.107:0/859251640 learned_addr learned my addr 172.21.15.107:0/859251640 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6000700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8104670 0x7fd6c81a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c7400700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8106b60 0x7fd6c819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 -- 172.21.15.107:0/859251640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8104670 msgr2=0x7fd6c81a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8104670 0x7fd6c81a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 -- 172.21.15.107:0/859251640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8106b60 msgr2=0x7fd6c819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8106b60 0x7fd6c819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 -- 172.21.15.107:0/859251640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0009c40 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.255+0000 7fd6c6a00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9c0 tx=0x7fd6b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.256+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8004a30 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.256+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6b800cca0 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.256+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8005580 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.256+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c819f500 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.256+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c81ab5a0 con 0x7fd6c8069e60 2024-03-11T19:49:05.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.257+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8101150 con 0x7fd6c8069e60 2024-03-11T19:49:05.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.261+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd6b8004c20 con 0x7fd6c8069e60 2024-03-11T19:49:05.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.262+0000 7fd6bf400700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 0x7fd6ac0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:05.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.262+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6b8014070 con 0x7fd6c8069e60 2024-03-11T19:49:05.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.263+0000 7fd6c6000700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 0x7fd6ac0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:05.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.263+0000 7fd6c6000700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 0x7fd6ac0a4000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6c819feb0 tx=0x7fd6b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.265+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6b8096820 con 0x7fd6c8069e60 2024-03-11T19:49:05.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.483+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6c8118310 con 0x7fd6c8069e60 2024-03-11T19:49:05.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.484+0000 7fd6bf400700 1 -- 172.21.15.107:0/859251640 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6b8099ee0 con 0x7fd6c8069e60 2024-03-11T19:49:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 msgr2=0x7fd6ac0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 0x7fd6ac0a4000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6c819feb0 tx=0x7fd6b4009260 comp rx=0 tx=0).stop 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 msgr2=0x7fd6c81a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9c0 tx=0x7fd6b800ecd0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 shutdown_connections 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6ac0a1b40 0x7fd6ac0a4000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c8106b60 0x7fd6c819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c8069e60 0x7fd6c81a4620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 --2- 172.21.15.107:0/859251640 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6c8104670 0x7fd6c81a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:05.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.486+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 >> 172.21.15.107:0/859251640 conn(0x7fd6c8077aa0 msgr2=0x7fd6c8103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:05.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.487+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 shutdown_connections 2024-03-11T19:49:05.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:05.487+0000 7fd6cdc00700 1 -- 172.21.15.107:0/859251640 wait complete. 2024-03-11T19:49:05.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:05 smithi107 ceph-mon[31712]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:05.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:05 smithi122 ceph-mon[36397]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:05.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:05 smithi145 ceph-mon[36730]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.a on smithi107 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.d on smithi107 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.g on smithi107 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.j on smithi107 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.b on smithi122 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.e on smithi122 2024-03-11T19:49:06.376 INFO:tasks.cephadm:Adding mds.h on smithi122 2024-03-11T19:49:06.377 INFO:tasks.cephadm:Adding mds.k on smithi122 2024-03-11T19:49:06.377 INFO:tasks.cephadm:Adding mds.c on smithi145 2024-03-11T19:49:06.377 INFO:tasks.cephadm:Adding mds.f on smithi145 2024-03-11T19:49:06.377 INFO:tasks.cephadm:Adding mds.i on smithi145 2024-03-11T19:49:06.377 INFO:tasks.cephadm:Adding mds.l on smithi145 2024-03-11T19:49:06.377 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l' 2024-03-11T19:49:06.640 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:49:06.679 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:06 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/859251640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:06.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:06 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/859251640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:06.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:06 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/859251640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:07.755 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:07 smithi145 ceph-mon[36730]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:07.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:07 smithi122 ceph-mon[36397]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:07.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:07 smithi107 ceph-mon[31712]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:08.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 -- 172.21.15.145:0/3911593182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 msgr2=0x7fc55c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 --2- 172.21.15.145:0/3911593182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c1020c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009990 tx=0x7fc54c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 -- 172.21.15.145:0/3911593182 shutdown_connections 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 --2- 172.21.15.145:0/3911593182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc55c103840 0x7fc55c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 --2- 172.21.15.145:0/3911593182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c1020c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 --2- 172.21.15.145:0/3911593182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 0x7fc55c103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 -- 172.21.15.145:0/3911593182 >> 172.21.15.145:0/3911593182 conn(0x7fc55c0fd280 msgr2=0x7fc55c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 -- 172.21.15.145:0/3911593182 shutdown_connections 2024-03-11T19:49:08.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.308+0000 7fc560d6d700 1 -- 172.21.15.145:0/3911593182 wait complete. 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 Processor -- start 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 -- start start 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 0x7fc55c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc55c103840 0x7fc55c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19f850 con 0x7fc55c103840 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19f990 con 0x7fc55c102ea0 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc560d6d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c19fb50 con 0x7fc55c101ca0 2024-03-11T19:49:08.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:58152/0 (socket says 172.21.15.145:58152) 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 -- 172.21.15.145:0/1838492403 learned_addr learned my addr 172.21.15.145:0/1838492403 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc559d9b700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 0x7fc55c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 -- 172.21.15.145:0/1838492403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 msgr2=0x7fc55c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 0x7fc55c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 -- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc55c103840 msgr2=0x7fc55c19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc55c103840 0x7fc55c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 -- 172.21.15.145:0/1838492403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0095f0 con 0x7fc55c101ca0 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.309+0000 7fc55a59c700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0095c0 tx=0x7fc54c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.310+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c01d070 con 0x7fc55c101ca0 2024-03-11T19:49:08.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.310+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c19fdd0 con 0x7fc55c101ca0 2024-03-11T19:49:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.310+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc54c00fc30 con 0x7fc55c101ca0 2024-03-11T19:49:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.310+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0173f0 con 0x7fc55c101ca0 2024-03-11T19:49:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.310+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c1a0210 con 0x7fc55c101ca0 2024-03-11T19:49:08.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.311+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c1090e0 con 0x7fc55c101ca0 2024-03-11T19:49:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.312+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc54c0039d0 con 0x7fc55c101ca0 2024-03-11T19:49:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.312+0000 7fc54b7fe700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 0x7fc5440a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.312+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc54c0ce370 con 0x7fc55c101ca0 2024-03-11T19:49:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.313+0000 7fc559d9b700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 0x7fc5440a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:08.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.314+0000 7fc559d9b700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 0x7fc5440a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc550005950 tx=0x7fc5500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:08.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.316+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc54c097ab0 con 0x7fc55c101ca0 2024-03-11T19:49:08.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.520+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc55c02d9e0 con 0x7fc5440a1ac0 2024-03-11T19:49:08.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.538+0000 7fc54b7fe700 1 -- 172.21.15.145:0/1838492403 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc55c02d9e0 con 0x7fc5440a1ac0 2024-03-11T19:49:08.541 INFO:teuthology.orchestra.run.smithi145.stdout:Scheduled mds.all update... 2024-03-11T19:49:08.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 msgr2=0x7fc5440a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 0x7fc5440a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc550005950 tx=0x7fc5500058e0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 msgr2=0x7fc55c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0095c0 tx=0x7fc54c00f740 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 shutdown_connections 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc5440a1ac0 0x7fc5440a3f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc55c103840 0x7fc55c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc55c101ca0 0x7fc55c197d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 --2- 172.21.15.145:0/1838492403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc55c102ea0 0x7fc55c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 >> 172.21.15.145:0/1838492403 conn(0x7fc55c0fd280 msgr2=0x7fc55c106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 shutdown_connections 2024-03-11T19:49:08.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-03-11T19:49:08.539+0000 7fc560d6d700 1 -- 172.21.15.145:0/1838492403 wait complete. 2024-03-11T19:49:09.135 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.a.service 2024-03-11T19:49:09.141 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.d.service 2024-03-11T19:49:09.187 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.g.service 2024-03-11T19:49:09.233 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.j.service 2024-03-11T19:49:09.279 DEBUG:teuthology.orchestra.run.smithi122:mds.b> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.b.service 2024-03-11T19:49:09.281 DEBUG:teuthology.orchestra.run.smithi122:mds.e> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.e.service 2024-03-11T19:49:09.324 DEBUG:teuthology.orchestra.run.smithi122:mds.h> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.h.service 2024-03-11T19:49:09.367 DEBUG:teuthology.orchestra.run.smithi122:mds.k> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.k.service 2024-03-11T19:49:09.410 DEBUG:teuthology.orchestra.run.smithi145:mds.c> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.c.service 2024-03-11T19:49:09.412 DEBUG:teuthology.orchestra.run.smithi145:mds.f> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.f.service 2024-03-11T19:49:09.455 DEBUG:teuthology.orchestra.run.smithi145:mds.i> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.i.service 2024-03-11T19:49:09.498 DEBUG:teuthology.orchestra.run.smithi145:mds.l> sudo journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.l.service 2024-03-11T19:49:09.541 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-11T19:49:09.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:49:09.548 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:09 smithi107 ceph-mon[31712]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:09.548 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:09.548 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:09 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.757+0000 7f118b200640 1 -- 172.21.15.107:0/133924876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 msgr2=0x7f1184102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.757+0000 7f118b200640 1 --2- 172.21.15.107:0/133924876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f1184102810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1170009900 tx=0x7f117002bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 -- 172.21.15.107:0/133924876 shutdown_connections 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 --2- 172.21.15.107:0/133924876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f1184102810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 --2- 172.21.15.107:0/133924876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1184102d50 0x7f11841095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 --2- 172.21.15.107:0/133924876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11841011a0 0x7f11841015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 -- 172.21.15.107:0/133924876 >> 172.21.15.107:0/133924876 conn(0x7f11840fc730 msgr2=0x7f11840feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 -- 172.21.15.107:0/133924876 shutdown_connections 2024-03-11T19:49:09.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.758+0000 7f118b200640 1 -- 172.21.15.107:0/133924876 wait complete. 2024-03-11T19:49:09.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:09 smithi122 ceph-mon[36397]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:09.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:09.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:09 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 Processor -- start 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 -- start start 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f11841999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 0x7f11841a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a0f50 con 0x7f11841011a0 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a10c0 con 0x7f1184102390 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.759+0000 7f118b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a1280 con 0x7f1184102d50 2024-03-11T19:49:09.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1183e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1183400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f11841999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 0x7f11841a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1183e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48220/0 (socket says 172.21.15.107:48220) 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1183e00640 1 -- 172.21.15.107:0/1173122956 learned_addr learned my addr 172.21.15.107:0/1173122956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 -- 172.21.15.107:0/1173122956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 msgr2=0x7f11841999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f11841999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 -- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 msgr2=0x7f1184199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 -- 172.21.15.107:0/1173122956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1170009560 con 0x7f1184102d50 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1183e00640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.760+0000 7f1189200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 0x7f11841a0a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f117c009fd0 tx=0x7f117c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.761+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00ce20 con 0x7f1184102d50 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.761+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f117c005020 con 0x7f1184102d50 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.761+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11841a1560 con 0x7f1184102d50 2024-03-11T19:49:09.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.761+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c010690 con 0x7f1184102d50 2024-03-11T19:49:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.761+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841a1a40 con 0x7f1184102d50 2024-03-11T19:49:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.762+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118410cc00 con 0x7f1184102d50 2024-03-11T19:49:09.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.763+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f117c004b20 con 0x7f1184102d50 2024-03-11T19:49:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.765+0000 7f1180c00640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 0x7f11680a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.765+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f117c014070 con 0x7f1184102d50 2024-03-11T19:49:09.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.765+0000 7f1183e00640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 0x7f11680a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:09.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.766+0000 7f1183e00640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 0x7f11680a3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11740058c0 tx=0x7f1174005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:09.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.769+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f117c096ce0 con 0x7f1184102d50 2024-03-11T19:49:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:09 smithi145 ceph-mon[36730]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:09 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:09.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.968+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11841a1ea0 con 0x7f1184102d50 2024-03-11T19:49:09.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.970+0000 7f1180c00640 1 -- 172.21.15.107:0/1173122956 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18548 (secure 0 0 0) 0x7f117c09a3a0 con 0x7f1184102d50 2024-03-11T19:49:09.975 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:09.975 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:48:58.522066+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:49:09.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 msgr2=0x7f11680a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:09.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 0x7f11680a3f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11740058c0 tx=0x7f1174005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:09.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 msgr2=0x7f11841a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:09.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 0x7f11841a0a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f117c009fd0 tx=0x7f117c00ef10 comp rx=0 tx=0).stop 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 shutdown_connections 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f11680a1ab0 0x7f11680a3f70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1184102390 0x7f11841999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1184102d50 0x7f11841a0a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 --2- 172.21.15.107:0/1173122956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11841011a0 0x7f1184199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.972+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 >> 172.21.15.107:0/1173122956 conn(0x7f11840fc730 msgr2=0x7f11840733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.973+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 shutdown_connections 2024-03-11T19:49:09.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:09.973+0000 7f118b200640 1 -- 172.21.15.107:0/1173122956 wait complete. 2024-03-11T19:49:09.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:49:09.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 -- 172.21.15.107:0/1406392994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102d40 msgr2=0x7f7bac1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/1406392994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102d40 0x7f7bac1095d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009ca0 tx=0x7f7ba402c100 comp rx=0 tx=0).stop 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 -- 172.21.15.107:0/1406392994 shutdown_connections 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/1406392994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102d40 0x7f7bac1095d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/1406392994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102380 0x7f7bac102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/1406392994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 -- 172.21.15.107:0/1406392994 >> 172.21.15.107:0/1406392994 conn(0x7f7bac0fc720 msgr2=0x7f7bac0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 -- 172.21.15.107:0/1406392994 shutdown_connections 2024-03-11T19:49:10.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.210+0000 7f7bb0c00640 1 -- 172.21.15.107:0/1406392994 wait complete. 2024-03-11T19:49:10.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.211+0000 7f7bb0c00640 1 Processor -- start 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.211+0000 7f7bb0c00640 1 -- start start 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102d40 0x7f7bac1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1a0ed0 con 0x7f7bac102d40 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1a1040 con 0x7f7bac101190 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7bb0c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1a1200 con 0x7f7bac102380 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52512/0 (socket says 172.21.15.107:52512) 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:42474/0 (socket says 172.21.15.107:42474) 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 -- 172.21.15.107:0/3931130953 learned_addr learned my addr 172.21.15.107:0/3931130953 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:10.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7baa800640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102d40 0x7f7bac1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 -- 172.21.15.107:0/3931130953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 msgr2=0x7f7bac1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 -- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102d40 msgr2=0x7f7bac1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102d40 0x7f7bac1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.212+0000 7f7ba9400640 1 -- 172.21.15.107:0/3931130953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4009900 con 0x7f7bac102380 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7ba9400640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00e9f0 tx=0x7f7b9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c009700 con 0x7f7bac102380 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c00cd80 con 0x7f7bac102380 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac1a14e0 con 0x7f7bac102380 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c018920 con 0x7f7bac102380 2024-03-11T19:49:10.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.213+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac1a1a40 con 0x7f7bac102380 2024-03-11T19:49:10.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.214+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac10cba0 con 0x7f7bac102380 2024-03-11T19:49:10.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.216+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7b9c018a80 con 0x7f7bac102380 2024-03-11T19:49:10.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.217+0000 7f7ba2a00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 0x7f7b900a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.217+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b9c014070 con 0x7f7bac102380 2024-03-11T19:49:10.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.217+0000 7f7ba9e00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 0x7f7b900a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.218+0000 7f7ba9e00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 0x7f7b900a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005fd0 tx=0x7f7b9400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.222+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b9c052d20 con 0x7f7bac102380 2024-03-11T19:49:10.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.420+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bac10cd30 con 0x7f7bac102380 2024-03-11T19:49:10.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.421+0000 7f7ba2a00640 1 -- 172.21.15.107:0/3931130953 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bac10cd30 con 0x7f7bac102380 2024-03-11T19:49:10.426 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:49:10.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 msgr2=0x7f7b900a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 0x7f7b900a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005fd0 tx=0x7f7b9400b410 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 msgr2=0x7f7bac199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00e9f0 tx=0x7f7b9c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 shutdown_connections 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b900a1ab0 0x7f7b900a3f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bac102d40 0x7f7bac1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7bac102380 0x7f7bac199920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 --2- 172.21.15.107:0/3931130953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bac101190 0x7f7bac1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.423+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 >> 172.21.15.107:0/3931130953 conn(0x7f7bac0fc720 msgr2=0x7f7bac0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.424+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 shutdown_connections 2024-03-11T19:49:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.424+0000 7f7bb0c00640 1 -- 172.21.15.107:0/3931130953 wait complete. 2024-03-11T19:49:10.449 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 -- 172.21.15.107:0/2068296632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 msgr2=0x7f2a78073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/2068296632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78073590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009860 tx=0x7f2a6802bc10 comp rx=0 tx=0).stop 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 -- 172.21.15.107:0/2068296632 shutdown_connections 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/2068296632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a78073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/2068296632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 0x7f2a7810b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/2068296632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78073590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 -- 172.21.15.107:0/2068296632 >> 172.21.15.107:0/2068296632 conn(0x7f2a780fc510 msgr2=0x7f2a780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:10.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 -- 172.21.15.107:0/2068296632 shutdown_connections 2024-03-11T19:49:10.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.658+0000 7f2a7cc00640 1 -- 172.21.15.107:0/2068296632 wait complete. 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 Processor -- start 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 -- start start 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 0x7f2a781a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a0dc0 con 0x7f2a78075140 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a0f30 con 0x7f2a78073b60 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.659+0000 7f2a7cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a10f0 con 0x7f2a78108e70 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:42502/0 (socket says 172.21.15.107:42502) 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 -- 172.21.15.107:0/1086737169 learned_addr learned my addr 172.21.15.107:0/1086737169 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77e00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 0x7f2a781a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 -- 172.21.15.107:0/1086737169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 msgr2=0x7f2a781a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 0x7f2a781a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 -- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 msgr2=0x7f2a78199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 -- 172.21.15.107:0/1086737169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70009900 con 0x7f2a78073b60 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.660+0000 7f2a77400640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802c150 tx=0x7f2a680346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.661+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68005460 con 0x7f2a78073b60 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.661+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a68005c90 con 0x7f2a78073b60 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.661+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a68009490 con 0x7f2a78073b60 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.661+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68004b60 con 0x7f2a78073b60 2024-03-11T19:49:10.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.661+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a781a1690 con 0x7f2a78073b60 2024-03-11T19:49:10.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.662+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a78107c30 con 0x7f2a78073b60 2024-03-11T19:49:10.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.664+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f2a68004d20 con 0x7f2a78073b60 2024-03-11T19:49:10.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.665+0000 7f2a74c00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 0x7f2a5c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:10.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.665+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a680ef060 con 0x7f2a78073b60 2024-03-11T19:49:10.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.665+0000 7f2a6ea00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 0x7f2a5c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:10.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.666+0000 7f2a6ea00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 0x7f2a5c0a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a7819a7e0 tx=0x7f2a60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:10.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.670+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a680b8670 con 0x7f2a78073b60 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.727 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:10 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1173122956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:10.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:10 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1173122956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:10.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:10 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1173122956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:10.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.911+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7802cdd0 con 0x7f2a78073b60 2024-03-11T19:49:10.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.912+0000 7f2a74c00640 1 -- 172.21.15.107:0/1086737169 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f2a680bbd30 con 0x7f2a78073b60 2024-03-11T19:49:10.917 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:10.917 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-11T19:49:10.917 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2024-03-11T19:49:10.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.914+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 msgr2=0x7f2a5c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.914+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 0x7f2a5c0a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a7819a7e0 tx=0x7f2a60005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:10.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.914+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 msgr2=0x7f2a781992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:10.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.914+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802c150 tx=0x7f2a680346f0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 shutdown_connections 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a5c0a1a30 0x7f2a5c0a3ef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a78108e70 0x7f2a781a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a78075140 0x7f2a78199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 --2- 172.21.15.107:0/1086737169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a78073b60 0x7f2a781992d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 >> 172.21.15.107:0/1086737169 conn(0x7f2a780fc510 msgr2=0x7f2a78104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 shutdown_connections 2024-03-11T19:49:10.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:10.915+0000 7f2a7cc00640 1 -- 172.21.15.107:0/1086737169 wait complete. 2024-03-11T19:49:11.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:11 smithi145 ceph-mon[36730]: Deploying daemon mds.l on smithi145 2024-03-11T19:49:11.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:11 smithi145 ceph-mon[36730]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:11.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:11 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3931130953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:11.636 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:11 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1086737169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:11 smithi122 ceph-mon[36397]: Deploying daemon mds.l on smithi145 2024-03-11T19:49:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:11 smithi122 ceph-mon[36397]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:11 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3931130953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:11 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1086737169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:11 smithi107 ceph-mon[31712]: Deploying daemon mds.l on smithi145 2024-03-11T19:49:11.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:11 smithi107 ceph-mon[31712]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:11.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3931130953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:11.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1086737169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:12.037 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:11 smithi145 systemd[1]: Starting Ceph mds.l for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:12.294 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 podman[66782]: 2024-03-11 19:49:11.937006165 +0000 UTC m=+0.017850828 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:12.295 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 podman[66782]: 2024-03-11 19:49:12.147200761 +0000 UTC m=+0.228045438 container create b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-11T19:49:12.787 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 podman[66782]: 2024-03-11 19:49:12.399858036 +0000 UTC m=+0.480702673 container init b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-11T19:49:12.787 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 podman[66782]: 2024-03-11 19:49:12.402730986 +0000 UTC m=+0.483575628 container start b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-11T19:49:12.788 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 ceph-mds[66801]: -- 172.21.15.145:0/794252363 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ab69a21a0 con 0x555ab6978800 2024-03-11T19:49:12.788 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l[66797]: starting mds.l at 2024-03-11T19:49:12.788 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 bash[66782]: b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b 2024-03-11T19:49:12.788 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 19:49:12 smithi145 systemd[1]: Started Ceph mds.l for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:12.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3695332710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d641045b0 msgr2=0x7f6d64104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3695332710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d641045b0 0x7f6d64104a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3695332710 shutdown_connections 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3695332710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d641045b0 0x7f6d64104a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3695332710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d64104f70 0x7f6d6410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3695332710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641033c0 0x7f6d641037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3695332710 >> 172.21.15.107:0/3695332710 conn(0x7f6d640fe970 msgr2=0x7f6d64100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:13.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3695332710 shutdown_connections 2024-03-11T19:49:13.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.154+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3695332710 wait complete. 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.155+0000 7f6d6ac00640 1 Processor -- start 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.155+0000 7f6d6ac00640 1 -- start start 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641045b0 0x7f6d64199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d64104f70 0x7f6d641a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:13.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641a0f20 con 0x7f6d641045b0 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641a1090 con 0x7f6d64104f70 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d6ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641a1250 con 0x7f6d641033c0 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d63e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d5be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641045b0 0x7f6d64199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d63e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52552/0 (socket says 172.21.15.107:52552) 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d63e00640 1 -- 172.21.15.107:0/3794138354 learned_addr learned my addr 172.21.15.107:0/3794138354 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.156+0000 7f6d68c00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d64104f70 0x7f6d641a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 -- 172.21.15.107:0/3794138354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d64104f70 msgr2=0x7f6d641a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d64104f70 0x7f6d641a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 -- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641045b0 msgr2=0x7f6d64199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:13.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641045b0 0x7f6d64199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 -- 172.21.15.107:0/3794138354 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c009560 con 0x7f6d641033c0 2024-03-11T19:49:13.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d63e00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9c0 tx=0x7f6d5400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:13.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54009700 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d5400cd80 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d641a1530 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54010430 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.157+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d641a1a10 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.158+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6410a6e0 con 0x7f6d641033c0 2024-03-11T19:49:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.160+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6d54003680 con 0x7f6d641033c0 2024-03-11T19:49:13.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.161+0000 7f6d61600640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 0x7f6d480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:13.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.162+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d54014070 con 0x7f6d641033c0 2024-03-11T19:49:13.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.162+0000 7f6d5be00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 0x7f6d480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:13.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.162+0000 7f6d5be00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 0x7f6d480a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c000c00 tx=0x7f6d4c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:13.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.165+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d540969c0 con 0x7f6d641033c0 2024-03-11T19:49:13.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.405+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d6410ee20 con 0x7f6d641033c0 2024-03-11T19:49:13.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.406+0000 7f6d61600640 1 -- 172.21.15.107:0/3794138354 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f6d5409a080 con 0x7f6d641033c0 2024-03-11T19:49:13.411 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:13.411 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:49:13.411 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 msgr2=0x7f6d480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 0x7f6d480a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c000c00 tx=0x7f6d4c02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 msgr2=0x7f6d64199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9c0 tx=0x7f6d5400ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 shutdown_connections 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6d480a1a30 0x7f6d480a3ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d64104f70 0x7f6d641a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d641045b0 0x7f6d64199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 --2- 172.21.15.107:0/3794138354 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d641033c0 0x7f6d64199430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:13.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 >> 172.21.15.107:0/3794138354 conn(0x7f6d640fe970 msgr2=0x7f6d641078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:13.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 shutdown_connections 2024-03-11T19:49:13.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:13.409+0000 7f6d6ac00640 1 -- 172.21.15.107:0/3794138354 wait complete. 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6832/1918081210,v1:172.21.15.145:6833/1918081210] up:boot 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: fsmap 1 up:standby 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:13.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:13 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3794138354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6832/1918081210,v1:172.21.15.145:6833/1918081210] up:boot 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: fsmap 1 up:standby 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:13 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3794138354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6832/1918081210,v1:172.21.15.145:6833/1918081210] up:boot 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: fsmap 1 up:standby 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:13.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:13 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3794138354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:14.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:14 smithi122 ceph-mon[36397]: Deploying daemon mds.k on smithi122 2024-03-11T19:49:14.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:14 smithi145 ceph-mon[36730]: Deploying daemon mds.k on smithi122 2024-03-11T19:49:14.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:14 smithi107 ceph-mon[31712]: Deploying daemon mds.k on smithi122 2024-03-11T19:49:14.926 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:14 smithi122 systemd[1]: Starting Ceph mds.k for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:15.263 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 podman[65976]: 2024-03-11 19:49:15.039968475 +0000 UTC m=+0.017555034 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:15.263 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 podman[65976]: 2024-03-11 19:49:15.153901264 +0000 UTC m=+0.131487801 container create aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:49:15.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 -- 172.21.15.107:0/2554375893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff441033c0 msgr2=0x7eff441037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 --2- 172.21.15.107:0/2554375893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff441033c0 0x7eff441037e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff38009900 tx=0x7eff3802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 -- 172.21.15.107:0/2554375893 shutdown_connections 2024-03-11T19:49:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 --2- 172.21.15.107:0/2554375893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff44104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 --2- 172.21.15.107:0/2554375893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff44104f70 0x7eff4410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 --2- 172.21.15.107:0/2554375893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff441033c0 0x7eff441037e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 -- 172.21.15.107:0/2554375893 >> 172.21.15.107:0/2554375893 conn(0x7eff440fe970 msgr2=0x7eff44100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 -- 172.21.15.107:0/2554375893 shutdown_connections 2024-03-11T19:49:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.652+0000 7eff4c600640 1 -- 172.21.15.107:0/2554375893 wait complete. 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.653+0000 7eff4c600640 1 Processor -- start 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 -- start start 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff441999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 0x7eff441a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff441a0f60 con 0x7eff44104f70 2024-03-11T19:49:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff441a10d0 con 0x7eff441045b0 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff441a1290 con 0x7eff441033c0 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff49200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff441999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff4a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 0x7eff441a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.654+0000 7eff49c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff441999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:45662/0 (socket says 172.21.15.107:45662) 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59586/0 (socket says 172.21.15.107:59586) 2024-03-11T19:49:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49200640 1 -- 172.21.15.107:0/3420185851 learned_addr learned my addr 172.21.15.107:0/3420185851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 -- 172.21.15.107:0/3420185851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 msgr2=0x7eff441999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff441999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 -- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 msgr2=0x7eff441a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 0x7eff441a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 -- 172.21.15.107:0/3420185851 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff38009560 con 0x7eff441033c0 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff4a600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 0x7eff441a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.655+0000 7eff49c00640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff38036040 tx=0x7eff38005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.656+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380043d0 con 0x7eff441033c0 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.656+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff441a1510 con 0x7eff441033c0 2024-03-11T19:49:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.656+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff380052d0 con 0x7eff441033c0 2024-03-11T19:49:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.656+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38003cd0 con 0x7eff441033c0 2024-03-11T19:49:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.656+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff441a1a40 con 0x7eff441033c0 2024-03-11T19:49:15.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.657+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4410a6e0 con 0x7eff441033c0 2024-03-11T19:49:15.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.659+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7eff38005440 con 0x7eff441033c0 2024-03-11T19:49:15.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.660+0000 7eff42000640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 0x7eff2c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:15.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.660+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff380ef680 con 0x7eff441033c0 2024-03-11T19:49:15.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.660+0000 7eff49200640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 0x7eff2c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:15.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.661+0000 7eff49200640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 0x7eff2c0a3f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff4419a980 tx=0x7eff30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.664+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff380b8c90 con 0x7eff441033c0 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:15 smithi122 ceph-mon[36397]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 podman[65976]: 2024-03-11 19:49:15.381990155 +0000 UTC m=+0.359576682 container init aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397) 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 podman[65976]: 2024-03-11 19:49:15.384689316 +0000 UTC m=+0.362275849 container start aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 ceph-mds[65995]: -- 172.21.15.122:0/418842904 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ca2f33e1a0 con 0x55ca2f315000 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k[65991]: starting mds.k at 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 bash[65976]: aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 2024-03-11T19:49:15.763 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 19:49:15 smithi122 systemd[1]: Started Ceph mds.k for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:15 smithi145 ceph-mon[36730]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:15 smithi107 ceph-mon[31712]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:15.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.906+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff4402cdd0 con 0x7eff441033c0 2024-03-11T19:49:15.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.907+0000 7eff42000640 1 -- 172.21.15.107:0/3420185851 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7eff380bc350 con 0x7eff441033c0 2024-03-11T19:49:15.912 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:15.912 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-11T19:49:15.912 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 msgr2=0x7eff2c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 0x7eff2c0a3f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff4419a980 tx=0x7eff30005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 msgr2=0x7eff44199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff38036040 tx=0x7eff38005e80 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 shutdown_connections 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7eff2c0a1ab0 0x7eff2c0a3f70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff441045b0 0x7eff441999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff44104f70 0x7eff441a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 --2- 172.21.15.107:0/3420185851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff441033c0 0x7eff44199470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:15.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 >> 172.21.15.107:0/3420185851 conn(0x7eff440fe970 msgr2=0x7eff441078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:15.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 shutdown_connections 2024-03-11T19:49:15.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:15.910+0000 7eff4c600640 1 -- 172.21.15.107:0/3420185851 wait complete. 2024-03-11T19:49:16.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6832/1820097321,v1:172.21.15.122:6833/1820097321] up:boot 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: fsmap 2 up:standby 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:16.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:16 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3420185851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6832/1820097321,v1:172.21.15.122:6833/1820097321] up:boot 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: fsmap 2 up:standby 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:16.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:16.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:16.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:16 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3420185851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6832/1820097321,v1:172.21.15.122:6833/1820097321] up:boot 2024-03-11T19:49:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: fsmap 2 up:standby 2024-03-11T19:49:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:16.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:16 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3420185851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:17 smithi122 ceph-mon[36397]: Deploying daemon mds.j on smithi107 2024-03-11T19:49:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:17 smithi122 ceph-mon[36397]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:17.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:17 smithi145 ceph-mon[36730]: Deploying daemon mds.j on smithi107 2024-03-11T19:49:17.790 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:17 smithi145 ceph-mon[36730]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:17 smithi107 ceph-mon[31712]: Deploying daemon mds.j on smithi107 2024-03-11T19:49:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:17 smithi107 ceph-mon[31712]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:17.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:18.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.375+0000 7faeb4600640 1 -- 172.21.15.107:0/1349281344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10d850 msgr2=0x7faeac10fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.375+0000 7faeb4600640 1 --2- 172.21.15.107:0/1349281344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10d850 0x7faeac10fd10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faea4005fd0 tx=0x7faea402cb60 comp rx=0 tx=0).stop 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 -- 172.21.15.107:0/1349281344 shutdown_connections 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 --2- 172.21.15.107:0/1349281344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10d850 0x7faeac10fd10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 --2- 172.21.15.107:0/1349281344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10adb0 0x7faeac10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 --2- 172.21.15.107:0/1349281344 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 -- 172.21.15.107:0/1349281344 >> 172.21.15.107:0/1349281344 conn(0x7faeac06dff0 msgr2=0x7faeac070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 -- 172.21.15.107:0/1349281344 shutdown_connections 2024-03-11T19:49:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.376+0000 7faeb4600640 1 -- 172.21.15.107:0/1349281344 wait complete. 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.377+0000 7faeb4600640 1 Processor -- start 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.377+0000 7faeb4600640 1 -- start start 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 0x7faeac10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10d850 0x7faeac10e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac10ea90 con 0x7faeac10d850 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac10ec00 con 0x7faeac10adb0 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb4600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac1a9ad0 con 0x7faeac1083f0 2024-03-11T19:49:18.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10d850 0x7faeac10e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 0x7faeac10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59596/0 (socket says 172.21.15.107:59596) 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.378+0000 7faeb1c00640 1 -- 172.21.15.107:0/4147744066 learned_addr learned my addr 172.21.15.107:0/4147744066 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 -- 172.21.15.107:0/4147744066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 msgr2=0x7faeac10f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 -- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10d850 msgr2=0x7faeac10e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10d850 0x7faeac10e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 -- 172.21.15.107:0/4147744066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40073b0 con 0x7faeac10adb0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb1200640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 0x7faeac10ddf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faea000e790 tx=0x7faea000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea000cc40 con 0x7faeac10adb0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faea000eea0 con 0x7faeac10adb0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0018680 con 0x7faeac10adb0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.379+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac1a9d80 con 0x7faeac10adb0 2024-03-11T19:49:18.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.380+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac1aa1e0 con 0x7faeac10adb0 2024-03-11T19:49:18.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.381+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac113190 con 0x7faeac10adb0 2024-03-11T19:49:18.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.383+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7faea00188a0 con 0x7faeac10adb0 2024-03-11T19:49:18.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.385+0000 7faeaa000640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 0x7fae940a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:18.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.385+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faea0014070 con 0x7faeac10adb0 2024-03-11T19:49:18.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.385+0000 7faeb1c00640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 0x7fae940a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:18.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.385+0000 7faeb1c00640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 0x7fae940a3f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae98009c90 tx=0x7fae98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:18.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.389+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faea0052d20 con 0x7faeac10adb0 2024-03-11T19:49:18.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.634+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeac113320 con 0x7faeac10adb0 2024-03-11T19:49:18.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.635+0000 7faeaa000640 1 -- 172.21.15.107:0/4147744066 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7faea00967e0 con 0x7faeac10adb0 2024-03-11T19:49:18.640 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:18.640 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-11T19:49:18.640 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2024-03-11T19:49:18.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 msgr2=0x7fae940a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:18.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 0x7fae940a3f40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae98009c90 tx=0x7fae98005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 msgr2=0x7faeac10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 0x7faeac10ddf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faea000e790 tx=0x7faea000ec50 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 shutdown_connections 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae940a1a80 0x7fae940a3f40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faeac10adb0 0x7faeac10ddf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeac10d850 0x7faeac10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 --2- 172.21.15.107:0/4147744066 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faeac1083f0 0x7faeac10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 >> 172.21.15.107:0/4147744066 conn(0x7faeac06dff0 msgr2=0x7faeac109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 shutdown_connections 2024-03-11T19:49:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:18.638+0000 7faeb4600640 1 -- 172.21.15.107:0/4147744066 wait complete. 2024-03-11T19:49:19.290 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 podman[71739]: 2024-03-11 19:49:19.095503325 +0000 UTC m=+0.293947058 container create 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-11T19:49:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:19 smithi122 ceph-mon[36397]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:19 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4147744066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:19 smithi145 ceph-mon[36730]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:19 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4147744066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:19 smithi107 ceph-mon[31712]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:19 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4147744066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 podman[71739]: 2024-03-11 19:49:19.393085267 +0000 UTC m=+0.591529006 container init 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux ) 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 podman[71739]: 2024-03-11 19:49:19.403487288 +0000 UTC m=+0.601931000 container start 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 ceph-mds[71758]: -- 172.21.15.107:0/905586454 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55736835a1a0 con 0x557368330800 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j[71753]: starting mds.j at 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 bash[71739]: 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 2024-03-11T19:49:19.795 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 19:49:19 smithi107 systemd[1]: Started Ceph mds.j for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:20.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6834/1132278897,v1:172.21.15.107:6835/1132278897] up:boot 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: fsmap 3 up:standby 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:20.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:20.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:20 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6834/1132278897,v1:172.21.15.107:6835/1132278897] up:boot 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: fsmap 3 up:standby 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:20.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:20 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:20.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6834/1132278897,v1:172.21.15.107:6835/1132278897] up:boot 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: fsmap 3 up:standby 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:20.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:20 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 -- 172.21.15.107:0/1649298632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341089b0 msgr2=0x7f2e34108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 --2- 172.21.15.107:0/1649298632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341089b0 0x7f2e34108e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009860 tx=0x7f2e2002bb70 comp rx=0 tx=0).stop 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 -- 172.21.15.107:0/1649298632 shutdown_connections 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 --2- 172.21.15.107:0/1649298632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341089b0 0x7f2e34108e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 --2- 172.21.15.107:0/1649298632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e34109370 0x7f2e3410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 --2- 172.21.15.107:0/1649298632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e341077c0 0x7f2e34107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 -- 172.21.15.107:0/1649298632 >> 172.21.15.107:0/1649298632 conn(0x7f2e34076790 msgr2=0x7f2e34078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 -- 172.21.15.107:0/1649298632 shutdown_connections 2024-03-11T19:49:20.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.854+0000 7f2e3c400640 1 -- 172.21.15.107:0/1649298632 wait complete. 2024-03-11T19:49:20.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 Processor -- start 2024-03-11T19:49:20.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 -- start start 2024-03-11T19:49:20.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 0x7f2e3419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:20.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e34109370 0x7f2e341a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e341a5360 con 0x7f2e341089b0 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e341a54d0 con 0x7f2e341077c0 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e3c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e341a5690 con 0x7f2e34109370 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e39a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 0x7f2e3419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.855+0000 7f2e39000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47342/0 (socket says 172.21.15.107:47342) 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 0x7f2e3419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:45714/0 (socket says 172.21.15.107:45714) 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 -- 172.21.15.107:0/3114196893 learned_addr learned my addr 172.21.15.107:0/3114196893 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e3a400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e34109370 0x7f2e341a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 -- 172.21.15.107:0/3114196893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e34109370 msgr2=0x7f2e341a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e34109370 0x7f2e341a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 -- 172.21.15.107:0/3114196893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 msgr2=0x7f2e3419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 0x7f2e3419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:20.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 -- 172.21.15.107:0/3114196893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c009900 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e39000640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005940 tx=0x7f2e200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.856+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2002c510 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.857+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e20009490 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.857+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e2002cb50 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.857+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20032860 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.857+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e341a5af0 con 0x7f2e341089b0 2024-03-11T19:49:20.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.858+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3410eae0 con 0x7f2e341089b0 2024-03-11T19:49:20.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.860+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f2e2003b040 con 0x7f2e341089b0 2024-03-11T19:49:20.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.861+0000 7f2e32000640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 0x7f2e1c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:20.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.861+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e200eec80 con 0x7f2e341089b0 2024-03-11T19:49:20.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.861+0000 7f2e39a00640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 0x7f2e1c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:20.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.862+0000 7f2e39a00640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 0x7f2e1c0a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e34108810 tx=0x7f2e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:20.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:20.866+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e200b8340 con 0x7f2e341089b0 2024-03-11T19:49:21.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.107+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e341131a0 con 0x7f2e341089b0 2024-03-11T19:49:21.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.108+0000 7f2e32000640 1 -- 172.21.15.107:0/3114196893 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7f2e200bba00 con 0x7f2e341089b0 2024-03-11T19:49:21.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-11T19:49:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-03-11T19:49:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 msgr2=0x7f2e1c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 0x7f2e1c0a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e34108810 tx=0x7f2e28005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 msgr2=0x7f2e3419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005940 tx=0x7f2e200045e0 comp rx=0 tx=0).stop 2024-03-11T19:49:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 shutdown_connections 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2e1c0a1a30 0x7f2e1c0a3ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2e34109370 0x7f2e341a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e341089b0 0x7f2e3419ddb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 --2- 172.21.15.107:0/3114196893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e341077c0 0x7f2e3419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.110+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 >> 172.21.15.107:0/3114196893 conn(0x7f2e34076790 msgr2=0x7f2e3410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.111+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 shutdown_connections 2024-03-11T19:49:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:21.111+0000 7f2e3c400640 1 -- 172.21.15.107:0/3114196893 wait complete. 2024-03-11T19:49:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:21 smithi122 ceph-mon[36397]: Deploying daemon mds.d on smithi107 2024-03-11T19:49:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:21 smithi122 ceph-mon[36397]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:21 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3114196893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:21 smithi145 ceph-mon[36730]: Deploying daemon mds.d on smithi107 2024-03-11T19:49:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:21 smithi145 ceph-mon[36730]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:21.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:21 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3114196893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:21 smithi107 ceph-mon[31712]: Deploying daemon mds.d on smithi107 2024-03-11T19:49:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:21 smithi107 ceph-mon[31712]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:21.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:21 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3114196893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:23.134 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:23.175 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:22 smithi107 systemd[1]: Starting Ceph mds.d for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:23.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 -- 172.21.15.107:0/1543887618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 msgr2=0x7f0c7410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:23.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 --2- 172.21.15.107:0/1543887618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c7410fc00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009ca0 tx=0x7f0c6c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 -- 172.21.15.107:0/1543887618 shutdown_connections 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 --2- 172.21.15.107:0/1543887618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c7410fc00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 --2- 172.21.15.107:0/1543887618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 0x7f0c74108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 --2- 172.21.15.107:0/1543887618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c74107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 -- 172.21.15.107:0/1543887618 >> 172.21.15.107:0/1543887618 conn(0x7f0c74076790 msgr2=0x7f0c74078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 -- 172.21.15.107:0/1543887618 shutdown_connections 2024-03-11T19:49:23.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.336+0000 7f0c7c400640 1 -- 172.21.15.107:0/1543887618 wait complete. 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 Processor -- start 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 -- start start 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 0x7f0c7419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c741a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c741a52c0 con 0x7f0c74109370 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c741a5430 con 0x7f0c741077c0 2024-03-11T19:49:23.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c741a55f0 con 0x7f0c741089b0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c79a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c7a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c741a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c79a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:45720/0 (socket says 172.21.15.107:45720) 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.338+0000 7f0c79a00640 1 -- 172.21.15.107:0/3217318027 learned_addr learned my addr 172.21.15.107:0/3217318027 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79000640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 0x7f0c7419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 -- 172.21.15.107:0/3217318027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 msgr2=0x7f0c7419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 0x7f0c7419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 -- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 msgr2=0x7f0c741a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c741a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 -- 172.21.15.107:0/3217318027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c009900 con 0x7f0c741077c0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c7a400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c741a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c79a00640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d950 tx=0x7f0c6800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68009c30 con 0x7f0c741077c0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c68009d90 con 0x7f0c741077c0 2024-03-11T19:49:23.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.339+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c741a58d0 con 0x7f0c741077c0 2024-03-11T19:49:23.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.340+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800f880 con 0x7f0c741077c0 2024-03-11T19:49:23.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.340+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c741a5de0 con 0x7f0c741077c0 2024-03-11T19:49:23.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.341+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7410ea80 con 0x7f0c741077c0 2024-03-11T19:49:23.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.343+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f0c6800f9e0 con 0x7f0c741077c0 2024-03-11T19:49:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.344+0000 7f0c72000640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 0x7f0c5c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.344+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c680cd1c0 con 0x7f0c741077c0 2024-03-11T19:49:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.344+0000 7f0c79000640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 0x7f0c5c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:23.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.345+0000 7f0c79000640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 0x7f0c5c0a3ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c7419ece0 tx=0x7f0c60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:23.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.349+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c680967d0 con 0x7f0c741077c0 2024-03-11T19:49:23.492 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 podman[72084]: 2024-03-11 19:49:23.076739202 +0000 UTC m=+0.035633850 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:23.493 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 podman[72084]: 2024-03-11 19:49:23.214749575 +0000 UTC m=+0.173644259 container create 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:49:23.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.600+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c741131c0 con 0x7f0c741077c0 2024-03-11T19:49:23.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.601+0000 7f0c72000640 1 -- 172.21.15.107:0/3217318027 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7f0c68099e90 con 0x7f0c741077c0 2024-03-11T19:49:23.605 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:23.606 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-11T19:49:23.606 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-03-11T19:49:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 msgr2=0x7f0c5c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 0x7f0c5c0a3ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c7419ece0 tx=0x7f0c60005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 msgr2=0x7f0c7419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d950 tx=0x7f0c6800de10 comp rx=0 tx=0).stop 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 shutdown_connections 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f0c5c0a1a30 0x7f0c5c0a3ef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c74109370 0x7f0c741a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0c741089b0 0x7f0c7419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 --2- 172.21.15.107:0/3217318027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c741077c0 0x7f0c7419d7d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.603+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 >> 172.21.15.107:0/3217318027 conn(0x7f0c74076790 msgr2=0x7f0c7410bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.604+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 shutdown_connections 2024-03-11T19:49:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:23.604+0000 7f0c7c400640 1 -- 172.21.15.107:0/3217318027 wait complete. 2024-03-11T19:49:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:23 smithi122 ceph-mon[36397]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:23 smithi145 ceph-mon[36730]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:23 smithi107 ceph-mon[31712]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 podman[72084]: 2024-03-11 19:49:23.538355324 +0000 UTC m=+0.497250058 container init 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, io.buildah.version=1.33.5, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 podman[72084]: 2024-03-11 19:49:23.547401197 +0000 UTC m=+0.506295851 container start 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 ceph-mds[72142]: -- 172.21.15.107:0/2026921856 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f87a0ac1a0 con 0x55f87a083000 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d[72138]: starting mds.d at 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 bash[72084]: 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 2024-03-11T19:49:23.795 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 19:49:23 smithi107 systemd[1]: Started Ceph mds.d for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3217318027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:24 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3217318027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:24.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:24 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:24.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3217318027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:24.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:24.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:24.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:24.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:24 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:25.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:25 smithi122 ceph-mon[36397]: Deploying daemon mds.e on smithi122 2024-03-11T19:49:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:25 smithi122 ceph-mon[36397]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:25 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6836/3292957465,v1:172.21.15.107:6837/3292957465] up:boot 2024-03-11T19:49:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:25 smithi122 ceph-mon[36397]: fsmap 4 up:standby 2024-03-11T19:49:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:25 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:49:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:25 smithi145 ceph-mon[36730]: Deploying daemon mds.e on smithi122 2024-03-11T19:49:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:25 smithi145 ceph-mon[36730]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:25 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6836/3292957465,v1:172.21.15.107:6837/3292957465] up:boot 2024-03-11T19:49:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:25 smithi145 ceph-mon[36730]: fsmap 4 up:standby 2024-03-11T19:49:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:25 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:49:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:25 smithi107 ceph-mon[31712]: Deploying daemon mds.e on smithi122 2024-03-11T19:49:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:25 smithi107 ceph-mon[31712]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:25 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6836/3292957465,v1:172.21.15.107:6837/3292957465] up:boot 2024-03-11T19:49:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:25 smithi107 ceph-mon[31712]: fsmap 4 up:standby 2024-03-11T19:49:25.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:25 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 -- 172.21.15.107:0/4009867728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674109020 msgr2=0x7f267410b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 --2- 172.21.15.107:0/4009867728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674109020 0x7f267410b4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f266c009ca0 tx=0x7f266c02c180 comp rx=0 tx=0).stop 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 -- 172.21.15.107:0/4009867728 shutdown_connections 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 --2- 172.21.15.107:0/4009867728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674109020 0x7f267410b4c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 --2- 172.21.15.107:0/4009867728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674073b80 0x7f2674073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 --2- 172.21.15.107:0/4009867728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674075160 0x7f26740735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 -- 172.21.15.107:0/4009867728 >> 172.21.15.107:0/4009867728 conn(0x7f26740fc760 msgr2=0x7f26740feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 -- 172.21.15.107:0/4009867728 shutdown_connections 2024-03-11T19:49:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.846+0000 7f2678e00640 1 -- 172.21.15.107:0/4009867728 wait complete. 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.847+0000 7f2678e00640 1 Processor -- start 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.847+0000 7f2678e00640 1 -- start start 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.847+0000 7f2678e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.847+0000 7f2678e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 0x7f26741998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2678e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674109020 0x7f26741a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2678e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26741a0e70 con 0x7f2674075160 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2678e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26741a0fe0 con 0x7f2674073b80 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2678e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26741a11a0 con 0x7f2674109020 2024-03-11T19:49:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2671e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2671e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:34842/0 (socket says 172.21.15.107:34842) 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2671e00640 1 -- 172.21.15.107:0/565319312 learned_addr learned my addr 172.21.15.107:0/565319312 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 0x7f26741998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2672800640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674109020 0x7f26741a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 -- 172.21.15.107:0/565319312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674109020 msgr2=0x7f26741a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674109020 0x7f26741a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 -- 172.21.15.107:0/565319312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 msgr2=0x7f2674199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f266be00640 1 -- 172.21.15.107:0/565319312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c009900 con 0x7f2674075160 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.848+0000 7f2671e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f266be00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 0x7f26741998c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e9f0 tx=0x7f265c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00ccc0 con 0x7f2674075160 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f265c00ce20 con 0x7f2674075160 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26741a1480 con 0x7f2674075160 2024-03-11T19:49:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c018920 con 0x7f2674075160 2024-03-11T19:49:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.849+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26741a1990 con 0x7f2674075160 2024-03-11T19:49:25.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.850+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2674107de0 con 0x7f2674075160 2024-03-11T19:49:25.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.852+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f265c018a80 con 0x7f2674075160 2024-03-11T19:49:25.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.854+0000 7f266aa00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 0x7f26580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:25.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.855+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f265c014070 con 0x7f2674075160 2024-03-11T19:49:25.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.855+0000 7f2671e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 0x7f26580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:25.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.855+0000 7f2671e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 0x7f26580a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2664005fd0 tx=0x7f266400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:25.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:25.859+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f265c052d20 con 0x7f2674075160 2024-03-11T19:49:26.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.094+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f267402ce40 con 0x7f2674075160 2024-03-11T19:49:26.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.095+0000 7f266aa00640 1 -- 172.21.15.107:0/565319312 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3710 (secure 0 0 0) 0x7f265c096940 con 0x7f2674075160 2024-03-11T19:49:26.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2024-03-11T19:49:26.100 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2024-03-11T19:49:26.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 msgr2=0x7f26580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:26.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 0x7f26580a3ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2664005fd0 tx=0x7f266400b480 comp rx=0 tx=0).stop 2024-03-11T19:49:26.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 msgr2=0x7f26741998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:26.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 0x7f26741998c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e9f0 tx=0x7f265c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:49:26.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 shutdown_connections 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f26580a1a30 0x7f26580a3ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2674109020 0x7f26741a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674075160 0x7f26741998c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 --2- 172.21.15.107:0/565319312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2674073b80 0x7f2674199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.097+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 >> 172.21.15.107:0/565319312 conn(0x7f26740fc760 msgr2=0x7f2674104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.098+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 shutdown_connections 2024-03-11T19:49:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:26.098+0000 7f2678e00640 1 -- 172.21.15.107:0/565319312 wait complete. 2024-03-11T19:49:26.263 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 systemd[1]: Starting Ceph mds.e for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:26 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/565319312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 podman[66295]: 2024-03-11 19:49:26.252876937 +0000 UTC m=+0.017374371 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 podman[66295]: 2024-03-11 19:49:26.352221427 +0000 UTC m=+0.116718847 container create 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 podman[66295]: 2024-03-11 19:49:26.571818168 +0000 UTC m=+0.336315588 container init 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 podman[66295]: 2024-03-11 19:49:26.574378947 +0000 UTC m=+0.338876370 container start 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 ceph-mds[66314]: -- 172.21.15.122:0/1759665119 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560f0dcba1a0 con 0x560f0dc91000 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e[66310]: starting mds.e at 2024-03-11T19:49:26.676 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 bash[66295]: 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 2024-03-11T19:49:26.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:26 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/565319312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:26 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/565319312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:27.012 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 19:49:26 smithi122 systemd[1]: Started Ceph mds.e for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:27.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:27.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.739 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.740 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:27.740 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:27.740 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:27 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:27 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:27 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:28.122 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.323+0000 7f8ff5a00640 1 -- 172.21.15.107:0/3782677723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 msgr2=0x7f8ff0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.323+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/3782677723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0102810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009900 tx=0x7f8fd802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 -- 172.21.15.107:0/3782677723 shutdown_connections 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/3782677723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff0102d50 0x7f8ff01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/3782677723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0102810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/3782677723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff01011a0 0x7f8ff01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 -- 172.21.15.107:0/3782677723 >> 172.21.15.107:0/3782677723 conn(0x7f8ff00fc730 msgr2=0x7f8ff00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 -- 172.21.15.107:0/3782677723 shutdown_connections 2024-03-11T19:49:28.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.324+0000 7f8ff5a00640 1 -- 172.21.15.107:0/3782677723 wait complete. 2024-03-11T19:49:28.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.325+0000 7f8ff5a00640 1 Processor -- start 2024-03-11T19:49:28.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.325+0000 7f8ff5a00640 1 -- start start 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 0x7f8ff0074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff0102d50 0x7f8ff0073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0074210 con 0x7f8ff01011a0 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0074380 con 0x7f8ff0102d50 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8ff5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01a12b0 con 0x7f8ff0102390 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8feea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 0x7f8ff0074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8feea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 0x7f8ff0074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45672/0 (socket says 172.21.15.107:45672) 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8feea00640 1 -- 172.21.15.107:0/2868460712 learned_addr learned my addr 172.21.15.107:0/2868460712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.326+0000 7f8fef400640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff0102d50 0x7f8ff0073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 -- 172.21.15.107:0/2868460712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff0102d50 msgr2=0x7f8ff0073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff0102d50 0x7f8ff0073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 -- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 msgr2=0x7f8ff0074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 0x7f8ff0074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 -- 172.21.15.107:0/2868460712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009560 con 0x7f8ff0102390 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7e00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0073570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8fd802c170 tx=0x7f8fd8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80043d0 con 0x7f8ff0102390 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fd80052d0 con 0x7f8ff0102390 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8003d00 con 0x7f8ff0102390 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff01a1500 con 0x7f8ff0102390 2024-03-11T19:49:28.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.327+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01a1a30 con 0x7f8ff0102390 2024-03-11T19:49:28.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.329+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff0108460 con 0x7f8ff0102390 2024-03-11T19:49:28.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.330+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f8fd8005440 con 0x7f8ff0102390 2024-03-11T19:49:28.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.332+0000 7f8fe7400640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 0x7f8fd40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.332+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fd80ef710 con 0x7f8ff0102390 2024-03-11T19:49:28.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.333+0000 7f8feea00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 0x7f8fd40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.333+0000 7f8feea00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 0x7f8fd40a3f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00058c0 tx=0x7f8fe0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.336+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fd80b8d20 con 0x7f8ff0102390 2024-03-11T19:49:28.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.569+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff010cc00 con 0x7f8ff0102390 2024-03-11T19:49:28.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.570+0000 7f8fe7400640 1 -- 172.21.15.107:0/2868460712 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4504 (secure 0 0 0) 0x7f8fd8003e60 con 0x7f8ff0102390 2024-03-11T19:49:28.575 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:28.575 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-11T19:49:28.575 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 msgr2=0x7f8fd40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 0x7f8fd40a3f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00058c0 tx=0x7f8fe0005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 msgr2=0x7f8ff0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0073570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8fd802c170 tx=0x7f8fd8005e80 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 shutdown_connections 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8fd40a1ab0 0x7f8fd40a3f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ff0102d50 0x7f8ff0073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ff0102390 0x7f8ff0073570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 --2- 172.21.15.107:0/2868460712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ff01011a0 0x7f8ff0074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 >> 172.21.15.107:0/2868460712 conn(0x7f8ff00fc730 msgr2=0x7f8ff0105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 shutdown_connections 2024-03-11T19:49:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:28.573+0000 7f8ff5a00640 1 -- 172.21.15.107:0/2868460712 wait complete. 2024-03-11T19:49:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:28 smithi145 ceph-mon[36730]: Deploying daemon mds.f on smithi145 2024-03-11T19:49:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:28 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6834/1916658439,v1:172.21.15.122:6835/1916658439] up:boot 2024-03-11T19:49:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:28 smithi145 ceph-mon[36730]: fsmap 5 up:standby 2024-03-11T19:49:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:28 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:49:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:28 smithi145 ceph-mon[36730]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:28 smithi107 ceph-mon[31712]: Deploying daemon mds.f on smithi145 2024-03-11T19:49:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:28 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6834/1916658439,v1:172.21.15.122:6835/1916658439] up:boot 2024-03-11T19:49:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:28 smithi107 ceph-mon[31712]: fsmap 5 up:standby 2024-03-11T19:49:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:28 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:49:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:28 smithi107 ceph-mon[31712]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:28 smithi122 ceph-mon[36397]: Deploying daemon mds.f on smithi145 2024-03-11T19:49:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:28 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6834/1916658439,v1:172.21.15.122:6835/1916658439] up:boot 2024-03-11T19:49:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:28 smithi122 ceph-mon[36397]: fsmap 5 up:standby 2024-03-11T19:49:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:28 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:49:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:28 smithi122 ceph-mon[36397]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:29.225 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 systemd[1]: Starting Ceph mds.f for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:29.483 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 podman[67093]: 2024-03-11 19:49:29.124060531 +0000 UTC m=+0.017642982 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:29.483 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 podman[67093]: 2024-03-11 19:49:29.234277997 +0000 UTC m=+0.127860481 container create 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-11T19:49:29.483 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 podman[67093]: 2024-03-11 19:49:29.453704295 +0000 UTC m=+0.347286743 container init 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-11T19:49:29.483 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 podman[67093]: 2024-03-11 19:49:29.456336262 +0000 UTC m=+0.349918707 container start 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T19:49:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:29 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2868460712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:29.788 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 ceph-mds[67112]: -- 172.21.15.145:0/2164341271 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc81f981a0 con 0x55bc81f6f000 2024-03-11T19:49:29.788 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: starting mds.f at 2024-03-11T19:49:29.788 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 bash[67093]: 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 2024-03-11T19:49:29.788 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 19:49:29 smithi145 systemd[1]: Started Ceph mds.f for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:29 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2868460712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:29 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2868460712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:30.598 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6834/3941680931,v1:172.21.15.145:6835/3941680931] up:boot 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: fsmap 6 up:standby 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: Deploying daemon mds.c on smithi145 2024-03-11T19:49:30.637 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:30 smithi145 ceph-mon[36730]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6834/3941680931,v1:172.21.15.145:6835/3941680931] up:boot 2024-03-11T19:49:30.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: fsmap 6 up:standby 2024-03-11T19:49:30.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:49:30.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: Deploying daemon mds.c on smithi145 2024-03-11T19:49:30.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:30 smithi107 ceph-mon[31712]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 -- 172.21.15.107:0/4231389395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641089b0 msgr2=0x7ff664108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 --2- 172.21.15.107:0/4231389395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641089b0 0x7ff664108e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff654009900 tx=0x7ff65402bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 -- 172.21.15.107:0/4231389395 shutdown_connections 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 --2- 172.21.15.107:0/4231389395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 0x7ff66410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 --2- 172.21.15.107:0/4231389395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641089b0 0x7ff664108e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 --2- 172.21.15.107:0/4231389395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641077c0 0x7ff664107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 -- 172.21.15.107:0/4231389395 >> 172.21.15.107:0/4231389395 conn(0x7ff664076790 msgr2=0x7ff664078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 -- 172.21.15.107:0/4231389395 shutdown_connections 2024-03-11T19:49:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.816+0000 7ff66c200640 1 -- 172.21.15.107:0/4231389395 wait complete. 2024-03-11T19:49:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 Processor -- start 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 -- start start 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641089b0 0x7ff66419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 0x7ff6641a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641a52e0 con 0x7ff664109370 2024-03-11T19:49:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641a5450 con 0x7ff6641089b0 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641a5610 con 0x7ff6641077c0 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff669800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff66a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 0x7ff6641a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff669800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:34872/0 (socket says 172.21.15.107:34872) 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff669800640 1 -- 172.21.15.107:0/2259016584 learned_addr learned my addr 172.21.15.107:0/2259016584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.817+0000 7ff668e00640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641089b0 0x7ff66419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 -- 172.21.15.107:0/2259016584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641089b0 msgr2=0x7ff66419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641089b0 0x7ff66419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 -- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 msgr2=0x7ff6641a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 0x7ff6641a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 -- 172.21.15.107:0/2259016584 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654009560 con 0x7ff6641077c0 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff669800640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff65800d770 tx=0x7ff65800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658009c30 con 0x7ff6641077c0 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff65800de80 con 0x7ff6641077c0 2024-03-11T19:49:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6641a58f0 con 0x7ff6641077c0 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658010460 con 0x7ff6641077c0 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.818+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6641a5e20 con 0x7ff6641077c0 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.820+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66410ea80 con 0x7ff6641077c0 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.821+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff658003f60 con 0x7ff6641077c0 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.823+0000 7ff662000640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 0x7ff64c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.823+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6580ccbe0 con 0x7ff6641077c0 2024-03-11T19:49:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.823+0000 7ff668e00640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 0x7ff64c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.824+0000 7ff668e00640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 0x7ff64c0a3f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff654009230 tx=0x7ff654005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:30.827+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6580962a0 con 0x7ff6641077c0 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6834/3941680931,v1:172.21.15.145:6835/3941680931] up:boot 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: fsmap 6 up:standby 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: Deploying daemon mds.c on smithi145 2024-03-11T19:49:31.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:30 smithi122 ceph-mon[36397]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:31.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.062+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66402ce70 con 0x7ff6641077c0 2024-03-11T19:49:31.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.063+0000 7ff662000640 1 -- 172.21.15.107:0/2259016584 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5298 (secure 0 0 0) 0x7ff658099960 con 0x7ff6641077c0 2024-03-11T19:49:31.069 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:31.069 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-11T19:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 7 2024-03-11T19:49:31.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 msgr2=0x7ff64c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:31.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 0x7ff64c0a3f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff654009230 tx=0x7ff654005fd0 comp rx=0 tx=0).stop 2024-03-11T19:49:31.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 msgr2=0x7ff66419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff65800d770 tx=0x7ff65800dc30 comp rx=0 tx=0).stop 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 shutdown_connections 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff64c0a1ab0 0x7ff64c0a3f70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641089b0 0x7ff66419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff664109370 0x7ff6641a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 --2- 172.21.15.107:0/2259016584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6641077c0 0x7ff66419d7f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 >> 172.21.15.107:0/2259016584 conn(0x7ff664076790 msgr2=0x7ff66410bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:31.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.066+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 shutdown_connections 2024-03-11T19:49:31.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:31.067+0000 7ff66c200640 1 -- 172.21.15.107:0/2259016584 wait complete. 2024-03-11T19:49:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:31 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2259016584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:32.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:31 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2259016584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:32.038 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:31 smithi145 systemd[1]: Starting Ceph mds.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:32.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:31 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2259016584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:32.372 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 podman[67401]: 2024-03-11 19:49:32.01309132 +0000 UTC m=+0.017833214 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:32.372 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 podman[67401]: 2024-03-11 19:49:32.131826445 +0000 UTC m=+0.136568329 container create f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:49:32.372 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 podman[67401]: 2024-03-11 19:49:32.368038938 +0000 UTC m=+0.372780838 container init f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-11T19:49:32.705 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 podman[67401]: 2024-03-11 19:49:32.371067363 +0000 UTC m=+0.375809249 container start f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-11T19:49:32.705 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mds[67420]: -- 172.21.15.145:0/3221552697 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a37cd01a0 con 0x564a37ca6800 2024-03-11T19:49:32.705 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: starting mds.c at 2024-03-11T19:49:32.705 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 bash[67401]: f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 2024-03-11T19:49:32.705 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 19:49:32 smithi145 systemd[1]: Started Ceph mds.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:32 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:33.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:32 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:32 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:33.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 -- 172.21.15.107:0/2376913144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 msgr2=0x7f335c1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 --2- 172.21.15.107:0/2376913144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1035f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f334c009900 tx=0x7f334c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 -- 172.21.15.107:0/2376913144 shutdown_connections 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 --2- 172.21.15.107:0/2376913144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c104d80 0x7f335c10b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 --2- 172.21.15.107:0/2376913144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c1043c0 0x7f335c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 --2- 172.21.15.107:0/2376913144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1035f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 -- 172.21.15.107:0/2376913144 >> 172.21.15.107:0/2376913144 conn(0x7f335c0fe760 msgr2=0x7f335c100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 -- 172.21.15.107:0/2376913144 shutdown_connections 2024-03-11T19:49:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.310+0000 7f335be00640 1 -- 172.21.15.107:0/2376913144 wait complete. 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 Processor -- start 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 -- start start 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c1043c0 0x7f335c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c104d80 0x7f335c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1a0d90 con 0x7f335c1031d0 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1a0f00 con 0x7f335c1043c0 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.311+0000 7f335be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1a10c0 con 0x7f335c104d80 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45702/0 (socket says 172.21.15.107:45702) 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 -- 172.21.15.107:0/3629399555 learned_addr learned my addr 172.21.15.107:0/3629399555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335a000640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c1043c0 0x7f335c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335b400640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c104d80 0x7f335c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 -- 172.21.15.107:0/3629399555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c104d80 msgr2=0x7f335c1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c104d80 0x7f335c1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 -- 172.21.15.107:0/3629399555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c1043c0 msgr2=0x7f335c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c1043c0 0x7f335c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 -- 172.21.15.107:0/3629399555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c009560 con 0x7f335c1031d0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f335aa00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f334c005ab0 tx=0x7f334c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.312+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c004aa0 con 0x7f335c1031d0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.313+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c1a1340 con 0x7f335c1031d0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.313+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f334c0052d0 con 0x7f335c1031d0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.313+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c003e80 con 0x7f335c1031d0 2024-03-11T19:49:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.313+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c1a1850 con 0x7f335c1031d0 2024-03-11T19:49:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.314+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c10a540 con 0x7f335c1031d0 2024-03-11T19:49:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.316+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f334c005440 con 0x7f335c1031d0 2024-03-11T19:49:33.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.317+0000 7f3353400640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 0x7f33400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:33.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.317+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f334c0ef230 con 0x7f335c1031d0 2024-03-11T19:49:33.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.317+0000 7f335a000640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 0x7f33400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:33.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.318+0000 7f335a000640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 0x7f33400a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f335c19a7b0 tx=0x7f3348005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:33.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.321+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f334c0b88e0 con 0x7f335c1031d0 2024-03-11T19:49:33.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.557+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f335c02cda0 con 0x7f335c1031d0 2024-03-11T19:49:33.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.558+0000 7f3353400640 1 -- 172.21.15.107:0/3629399555 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6092 (secure 0 0 0) 0x7f334c0bbfa0 con 0x7f335c1031d0 2024-03-11T19:49:33.563 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:33.563 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-11T19:49:33.563 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-03-11T19:49:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.560+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 msgr2=0x7f33400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 0x7f33400a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f335c19a7b0 tx=0x7f3348005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 msgr2=0x7f335c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:33.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f334c005ab0 tx=0x7f334c004270 comp rx=0 tx=0).stop 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 shutdown_connections 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f33400a1a30 0x7f33400a3ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c1043c0 0x7f335c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f335c104d80 0x7f335c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 --2- 172.21.15.107:0/3629399555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f335c1031d0 0x7f335c1992a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 >> 172.21.15.107:0/3629399555 conn(0x7f335c0fe760 msgr2=0x7f335c107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:33.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 shutdown_connections 2024-03-11T19:49:33.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:33.561+0000 7f335be00640 1 -- 172.21.15.107:0/3629399555 wait complete. 2024-03-11T19:49:33.975 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:33 smithi122 ceph-mon[36397]: Deploying daemon mds.h on smithi122 2024-03-11T19:49:33.975 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:33 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6836/2305866182,v1:172.21.15.145:6837/2305866182] up:boot 2024-03-11T19:49:33.975 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:33 smithi122 ceph-mon[36397]: fsmap 7 up:standby 2024-03-11T19:49:33.975 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:49:33.975 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:33 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3629399555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:34.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:33 smithi145 ceph-mon[36730]: Deploying daemon mds.h on smithi122 2024-03-11T19:49:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:33 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6836/2305866182,v1:172.21.15.145:6837/2305866182] up:boot 2024-03-11T19:49:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:33 smithi145 ceph-mon[36730]: fsmap 7 up:standby 2024-03-11T19:49:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:49:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:33 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3629399555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:33 smithi107 ceph-mon[31712]: Deploying daemon mds.h on smithi122 2024-03-11T19:49:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:33 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6836/2305866182,v1:172.21.15.145:6837/2305866182] up:boot 2024-03-11T19:49:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:33 smithi107 ceph-mon[31712]: fsmap 7 up:standby 2024-03-11T19:49:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:49:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:33 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3629399555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:34.974 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:34 smithi122 ceph-mon[36397]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:34.975 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:34 smithi122 systemd[1]: Starting Ceph mds.h for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:34 smithi145 ceph-mon[36730]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:35.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:34 smithi107 ceph-mon[31712]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:35.239 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:34 smithi122 podman[66604]: 2024-03-11 19:49:34.873366719 +0000 UTC m=+0.017057591 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:35.239 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:34 smithi122 podman[66604]: 2024-03-11 19:49:34.991397192 +0000 UTC m=+0.135088039 container create 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:49:35.239 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 podman[66604]: 2024-03-11 19:49:35.236168225 +0000 UTC m=+0.379859075 container init 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:49:35.513 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 podman[66604]: 2024-03-11 19:49:35.238917012 +0000 UTC m=+0.382607867 container start 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True) 2024-03-11T19:49:35.513 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 ceph-mds[66624]: -- 172.21.15.122:0/1914649187 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fa9f561a0 con 0x560fa9f2d000 2024-03-11T19:49:35.513 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: starting mds.h at 2024-03-11T19:49:35.513 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 bash[66604]: 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 2024-03-11T19:49:35.513 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 19:49:35 smithi122 systemd[1]: Started Ceph mds.h for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:35.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:35.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 -- 172.21.15.107:0/187619481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 msgr2=0x7fe4d81035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:35.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 --2- 172.21.15.107:0/187619481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d81035f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009860 tx=0x7fe4c802bb70 comp rx=0 tx=0).stop 2024-03-11T19:49:35.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 -- 172.21.15.107:0/187619481 shutdown_connections 2024-03-11T19:49:35.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 --2- 172.21.15.107:0/187619481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d8104d80 0x7fe4d810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 --2- 172.21.15.107:0/187619481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d81043c0 0x7fe4d8104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 --2- 172.21.15.107:0/187619481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d81035f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 -- 172.21.15.107:0/187619481 >> 172.21.15.107:0/187619481 conn(0x7fe4d80fe760 msgr2=0x7fe4d8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 -- 172.21.15.107:0/187619481 shutdown_connections 2024-03-11T19:49:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.802+0000 7fe4de400640 1 -- 172.21.15.107:0/187619481 wait complete. 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.803+0000 7fe4de400640 1 Processor -- start 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 -- start start 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8104d80 0x7fe4d81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81a0d40 con 0x7fe4d81043c0 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81a0eb0 con 0x7fe4d81031d0 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4de400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81a1070 con 0x7fe4d8104d80 2024-03-11T19:49:35.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39850/0 (socket says 172.21.15.107:39850) 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4cfe00640 1 -- 172.21.15.107:0/1220201781 learned_addr learned my addr 172.21.15.107:0/1220201781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4dda00640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8104d80 0x7fe4d81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.804+0000 7fe4dd000640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 -- 172.21.15.107:0/1220201781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8104d80 msgr2=0x7fe4d81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8104d80 0x7fe4d81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 -- 172.21.15.107:0/1220201781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 msgr2=0x7fe4d8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 -- 172.21.15.107:0/1220201781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0009560 con 0x7fe4d81043c0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4cfe00640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000e9c0 tx=0x7fe4c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000cde0 con 0x7fe4d81043c0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c8009490 con 0x7fe4d81043c0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c00035f0 con 0x7fe4d81043c0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.805+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0010650 con 0x7fe4d81043c0 2024-03-11T19:49:35.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.806+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d81a1720 con 0x7fe4d81043c0 2024-03-11T19:49:35.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.807+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d810a570 con 0x7fe4d81043c0 2024-03-11T19:49:35.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.808+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe4c0003ad0 con 0x7fe4d81043c0 2024-03-11T19:49:35.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.810+0000 7fe4d6000640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 0x7fe4bc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:35.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.810+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4c0014070 con 0x7fe4d81043c0 2024-03-11T19:49:35.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.810+0000 7fe4dd000640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 0x7fe4bc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:35.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.811+0000 7fe4dd000640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 0x7fe4bc0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8034b10 tx=0x7fe4c8034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:35.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:35.814+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4c0096a30 con 0x7fe4d81043c0 2024-03-11T19:49:36.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.053+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4d81a1350 con 0x7fe4d81043c0 2024-03-11T19:49:36.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.054+0000 7fe4d6000640 1 -- 172.21.15.107:0/1220201781 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6883 (secure 0 0 0) 0x7fe4d81a1350 con 0x7fe4d81043c0 2024-03-11T19:49:36.059 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:36.059 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-11T19:49:36.060 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 msgr2=0x7fe4bc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 0x7fe4bc0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8034b10 tx=0x7fe4c8034be0 comp rx=0 tx=0).stop 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 msgr2=0x7fe4d8199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000e9c0 tx=0x7fe4c000ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 shutdown_connections 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe4bc0a1a30 0x7fe4bc0a3ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe4d8104d80 0x7fe4d81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d81043c0 0x7fe4d8199790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 --2- 172.21.15.107:0/1220201781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4d81031d0 0x7fe4d8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:36.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 >> 172.21.15.107:0/1220201781 conn(0x7fe4d80fe760 msgr2=0x7fe4d8107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:36.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 shutdown_connections 2024-03-11T19:49:36.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:36.057+0000 7fe4de400640 1 -- 172.21.15.107:0/1220201781 wait complete. 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: Deploying daemon mds.i on smithi145 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6836/547863696,v1:172.21.15.122:6837/547863696] up:boot 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: fsmap 8 up:standby 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:49:36.619 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:36 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1220201781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: Deploying daemon mds.i on smithi145 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6836/547863696,v1:172.21.15.122:6837/547863696] up:boot 2024-03-11T19:49:36.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: fsmap 8 up:standby 2024-03-11T19:49:36.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:49:36.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:36 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1220201781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: Deploying daemon mds.i on smithi145 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6836/547863696,v1:172.21.15.122:6837/547863696] up:boot 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: fsmap 8 up:standby 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:49:36.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:36 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1220201781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:37.703 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:37 smithi145 ceph-mon[36730]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:37 smithi122 ceph-mon[36397]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:37.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:37 smithi107 ceph-mon[31712]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:38.038 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:37 smithi145 podman[67724]: 2024-03-11 19:49:37.937961368 +0000 UTC m=+0.138612926 container create c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-11T19:49:38.081 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 -- 172.21.15.107:0/2739184698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc100fc0 msgr2=0x7f3dcc1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/2739184698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc1013e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009900 tx=0x7f3dbc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 -- 172.21.15.107:0/2739184698 shutdown_connections 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/2739184698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc1021b0 0x7f3dcc102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/2739184698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc102b70 0x7f3dcc109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/2739184698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc1013e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 -- 172.21.15.107:0/2739184698 >> 172.21.15.107:0/2739184698 conn(0x7f3dcc0fc510 msgr2=0x7f3dcc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 -- 172.21.15.107:0/2739184698 shutdown_connections 2024-03-11T19:49:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.298+0000 7f3dd1c00640 1 -- 172.21.15.107:0/2739184698 wait complete. 2024-03-11T19:49:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 Processor -- start 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 -- start start 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc1021b0 0x7f3dcc1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc102b70 0x7f3dcc1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1a0d60 con 0x7f3dcc1021b0 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1a0ed0 con 0x7f3dcc102b70 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.299+0000 7f3dd1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1a1090 con 0x7f3dcc100fc0 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dc3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc1021b0 0x7f3dcc1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:39356/0 (socket says 172.21.15.107:39356) 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 -- 172.21.15.107:0/3758875622 learned_addr learned my addr 172.21.15.107:0/3758875622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc102b70 0x7f3dcc1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 -- 172.21.15.107:0/3758875622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc102b70 msgr2=0x7f3dcc1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc102b70 0x7f3dcc1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 -- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc1021b0 msgr2=0x7f3dcc1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc1021b0 0x7f3dcc1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 -- 172.21.15.107:0/3758875622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009560 con 0x7f3dcc100fc0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.300+0000 7f3dcbe00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005ab0 tx=0x7f3dbc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.301+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc004aa0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.301+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dbc0052d0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.301+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc003e80 con 0x7f3dcc100fc0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.301+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc1a1310 con 0x7f3dcc100fc0 2024-03-11T19:49:38.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.301+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc1a17d0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.302+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc10c9b0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.304+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3dbc005440 con 0x7f3dcc100fc0 2024-03-11T19:49:38.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.305+0000 7f3dc9600640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 0x7f3db00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:38.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.305+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3dbc0ef3c0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.305+0000 7f3dc3e00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 0x7f3db00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:38.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.306+0000 7f3dc3e00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 0x7f3db00a3f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc19a780 tx=0x7f3db400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:38.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.313+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3dbc0b89d0 con 0x7f3dcc100fc0 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 podman[67724]: 2024-03-11 19:49:38.165518599 +0000 UTC m=+0.366170173 container init c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 podman[67724]: 2024-03-11 19:49:38.168656178 +0000 UTC m=+0.369307743 container start c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 ceph-mds[67743]: -- 172.21.15.145:0/611497004 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596aa0401a0 con 0x5596aa016800 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: starting mds.i at 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 bash[67724]: c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b 2024-03-11T19:49:38.538 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 19:49:38 smithi145 systemd[1]: Started Ceph mds.i for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:38.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.555+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dcc10cb90 con 0x7f3dcc100fc0 2024-03-11T19:49:38.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.556+0000 7f3dc9600640 1 -- 172.21.15.107:0/3758875622 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7f3dbc0bc090 con 0x7f3dcc100fc0 2024-03-11T19:49:38.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:38.561 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:49:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2024-03-11T19:49:38.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 msgr2=0x7f3db00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:38.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 0x7f3db00a3f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc19a780 tx=0x7f3db400b390 comp rx=0 tx=0).stop 2024-03-11T19:49:38.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 msgr2=0x7f3dcc199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005ab0 tx=0x7f3dbc004270 comp rx=0 tx=0).stop 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 shutdown_connections 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3db00a1ab0 0x7f3db00a3f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dcc102b70 0x7f3dcc1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc1021b0 0x7f3dcc1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 --2- 172.21.15.107:0/3758875622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3dcc100fc0 0x7f3dcc199270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.559+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 >> 172.21.15.107:0/3758875622 conn(0x7f3dcc0fc510 msgr2=0x7f3dcc0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.560+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 shutdown_connections 2024-03-11T19:49:38.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:38.560+0000 7f3dd1c00640 1 -- 172.21.15.107:0/3758875622 wait complete. 2024-03-11T19:49:39.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:39.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: Deploying daemon mds.g on smithi107 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:boot 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: fsmap 9 up:standby 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:49:39.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:39 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3758875622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: Deploying daemon mds.g on smithi107 2024-03-11T19:49:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:boot 2024-03-11T19:49:39.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: fsmap 9 up:standby 2024-03-11T19:49:39.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:49:39.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:39 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3758875622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: Deploying daemon mds.g on smithi107 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:boot 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: fsmap 9 up:standby 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:49:39.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:39 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3758875622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:40.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:40.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 -- 172.21.15.107:0/45238291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914077720 msgr2=0x7ff914077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:40.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 --2- 172.21.15.107:0/45238291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914077720 0x7ff914077fc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009be0 tx=0x7ff90c02bef0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 -- 172.21.15.107:0/45238291 shutdown_connections 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 --2- 172.21.15.107:0/45238291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914077720 0x7ff914077fc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 --2- 172.21.15.107:0/45238291 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914076d80 0x7ff9140771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 --2- 172.21.15.107:0/45238291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff914075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.807+0000 7ff91b000640 1 -- 172.21.15.107:0/45238291 >> 172.21.15.107:0/45238291 conn(0x7ff9140fe970 msgr2=0x7ff914100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.808+0000 7ff91b000640 1 -- 172.21.15.107:0/45238291 shutdown_connections 2024-03-11T19:49:40.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.808+0000 7ff91b000640 1 -- 172.21.15.107:0/45238291 wait complete. 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.808+0000 7ff91b000640 1 Processor -- start 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 -- start start 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914076d80 0x7ff91419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914077720 0x7ff9141a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:40.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141a5290 con 0x7ff914076d80 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141a5400 con 0x7ff914075b40 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff91b000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141a55c0 con 0x7ff914077720 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff913e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff913e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:46756/0 (socket says 172.21.15.107:46756) 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff913400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914076d80 0x7ff91419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff913e00640 1 -- 172.21.15.107:0/1864301769 learned_addr learned my addr 172.21.15.107:0/1864301769 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.809+0000 7ff919000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914077720 0x7ff9141a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 -- 172.21.15.107:0/1864301769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914077720 msgr2=0x7ff9141a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914077720 0x7ff9141a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 -- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914076d80 msgr2=0x7ff91419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914076d80 0x7ff91419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 -- 172.21.15.107:0/1864301769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc009560 con 0x7ff914075b40 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff913e00640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff90400e9f0 tx=0x7ff90400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904009700 con 0x7ff914075b40 2024-03-11T19:49:40.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c009840 con 0x7ff914075b40 2024-03-11T19:49:40.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff90400cd80 con 0x7ff914075b40 2024-03-11T19:49:40.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904018920 con 0x7ff914075b40 2024-03-11T19:49:40.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.810+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141a5c70 con 0x7ff914075b40 2024-03-11T19:49:40.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.812+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91410fe40 con 0x7ff914075b40 2024-03-11T19:49:40.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.813+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff904018a80 con 0x7ff914075b40 2024-03-11T19:49:40.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.815+0000 7ff910c00640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 0x7ff8f80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:40.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.815+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff904014070 con 0x7ff914075b40 2024-03-11T19:49:40.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.815+0000 7ff913400640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 0x7ff8f80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:40.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.816+0000 7ff913400640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 0x7ff8f80a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff91419ecb0 tx=0x7ff8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:40.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:40.819+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff904052d20 con 0x7ff914075b40 2024-03-11T19:49:41.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.080+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff914113210 con 0x7ff914075b40 2024-03-11T19:49:41.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.082+0000 7ff910c00640 1 -- 172.21.15.107:0/1864301769 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7ff904096a30 con 0x7ff914075b40 2024-03-11T19:49:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:41.086 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:49:41.086 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 msgr2=0x7ff8f80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 0x7ff8f80a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff91419ecb0 tx=0x7ff8fc009260 comp rx=0 tx=0).stop 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 msgr2=0x7ff91419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff90400e9f0 tx=0x7ff90400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 shutdown_connections 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8f80a1a30 0x7ff8f80a3ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:41.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff914077720 0x7ff9141a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:41.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff914076d80 0x7ff91419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:41.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 --2- 172.21.15.107:0/1864301769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff914075b40 0x7ff91419d7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:41.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.085+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 >> 172.21.15.107:0/1864301769 conn(0x7ff9140fe970 msgr2=0x7ff9140fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:41.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.086+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 shutdown_connections 2024-03-11T19:49:41.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:41.086+0000 7ff91b000640 1 -- 172.21.15.107:0/1864301769 wait complete. 2024-03-11T19:49:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:41 smithi107 ceph-mon[31712]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:41.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:41 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1864301769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:41 smithi122 ceph-mon[36397]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:41 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1864301769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:41 smithi145 ceph-mon[36730]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:41 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1864301769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:42.194 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:41 smithi107 podman[72722]: 2024-03-11 19:49:41.88311187 +0000 UTC m=+0.170237505 container create 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-11T19:49:42.194 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 podman[72722]: 2024-03-11 19:49:42.189309875 +0000 UTC m=+0.476435490 container init 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-11T19:49:42.544 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 podman[72722]: 2024-03-11 19:49:42.201147725 +0000 UTC m=+0.488273299 container start 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-11T19:49:42.544 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 ceph-mds[72748]: -- 172.21.15.107:0/4218455645 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be7223e1a0 con 0x55be72214800 2024-03-11T19:49:42.544 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g[72742]: starting mds.g at 2024-03-11T19:49:42.544 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 bash[72722]: 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad 2024-03-11T19:49:42.544 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 19:49:42 smithi107 systemd[1]: Started Ceph mds.g for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:43.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:43.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 -- 172.21.15.107:0/2349452904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 msgr2=0x7f20e8100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:43.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 --2- 172.21.15.107:0/2349452904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8100850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d402bcd0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 -- 172.21.15.107:0/2349452904 shutdown_connections 2024-03-11T19:49:43.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 --2- 172.21.15.107:0/2349452904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e8100d90 0x7f20e8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 --2- 172.21.15.107:0/2349452904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8100850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 --2- 172.21.15.107:0/2349452904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e80ffa90 0x7f20e80ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 -- 172.21.15.107:0/2349452904 >> 172.21.15.107:0/2349452904 conn(0x7f20e80fb610 msgr2=0x7f20e80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:43.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 -- 172.21.15.107:0/2349452904 shutdown_connections 2024-03-11T19:49:43.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.302+0000 7f20ed800640 1 -- 172.21.15.107:0/2349452904 wait complete. 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.303+0000 7f20ed800640 1 Processor -- start 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.303+0000 7f20ed800640 1 -- start start 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e80ffa90 0x7f20e8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e8100d90 0x7f20e81a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81a0f00 con 0x7f20e81003f0 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81a1070 con 0x7f20e8100d90 2024-03-11T19:49:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20ed800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e81a1230 con 0x7f20e80ffa90 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39950/0 (socket says 172.21.15.107:39950) 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 -- 172.21.15.107:0/423737025 learned_addr learned my addr 172.21.15.107:0/423737025 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6a00640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e80ffa90 0x7f20e8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e7400640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e8100d90 0x7f20e81a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 -- 172.21.15.107:0/423737025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e80ffa90 msgr2=0x7f20e8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e80ffa90 0x7f20e8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 -- 172.21.15.107:0/423737025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e8100d90 msgr2=0x7f20e81a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e8100d90 0x7f20e81a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 -- 172.21.15.107:0/423737025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009560 con 0x7f20e81003f0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.304+0000 7f20e6000640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20d4000c00 tx=0x7f20d4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.305+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d402dd20 con 0x7f20e81003f0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.305+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20d402de80 con 0x7f20e81003f0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.305+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e81a14b0 con 0x7f20e81003f0 2024-03-11T19:49:43.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.305+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003d00 con 0x7f20e81003f0 2024-03-11T19:49:43.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.305+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e81a1a10 con 0x7f20e81003f0 2024-03-11T19:49:43.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.306+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e810a930 con 0x7f20e81003f0 2024-03-11T19:49:43.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.308+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f20d4004df0 con 0x7f20e81003f0 2024-03-11T19:49:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.309+0000 7f20df400640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 0x7f20cc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:43.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.309+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20d40ef780 con 0x7f20e81003f0 2024-03-11T19:49:43.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.309+0000 7f20e6a00640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 0x7f20cc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:43.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.310+0000 7f20e6a00640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 0x7f20cc0a3f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009b60 tx=0x7f20d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:43.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.313+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20d40b8eb0 con 0x7f20e81003f0 2024-03-11T19:49:43.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6838/988917516,v1:172.21.15.107:6839/988917516] up:boot 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: fsmap 10 up:standby 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:43.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:43 smithi107 ceph-mon[31712]: Deploying daemon mds.a on smithi107 2024-03-11T19:49:43.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.551+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e802cdd0 con 0x7f20e81003f0 2024-03-11T19:49:43.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.553+0000 7f20df400640 1 -- 172.21.15.107:0/423737025 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7f20d4003e60 con 0x7f20e81003f0 2024-03-11T19:49:43.557 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:43.557 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:49:43.557 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 msgr2=0x7f20cc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 0x7f20cc0a3f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009b60 tx=0x7f20d8005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 msgr2=0x7f20e8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20d4000c00 tx=0x7f20d4005e30 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 shutdown_connections 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f20cc0a1ab0 0x7f20cc0a3f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e8100d90 0x7f20e81a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e81003f0 0x7f20e8199950 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 --2- 172.21.15.107:0/423737025 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20e80ffa90 0x7f20e8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 >> 172.21.15.107:0/423737025 conn(0x7f20e80fb610 msgr2=0x7f20e80fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 shutdown_connections 2024-03-11T19:49:43.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:43.555+0000 7f20ed800640 1 -- 172.21.15.107:0/423737025 wait complete. 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6838/988917516,v1:172.21.15.107:6839/988917516] up:boot 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: fsmap 10 up:standby 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:43.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:43.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:43.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:43 smithi122 ceph-mon[36397]: Deploying daemon mds.a on smithi107 2024-03-11T19:49:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6838/988917516,v1:172.21.15.107:6839/988917516] up:boot 2024-03-11T19:49:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: fsmap 10 up:standby 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:43.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:43 smithi145 ceph-mon[36730]: Deploying daemon mds.a on smithi107 2024-03-11T19:49:44.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:44 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/423737025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:44.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:44 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/423737025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:44.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:44 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/423737025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:45.577 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:45.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:45 smithi107 ceph-mon[31712]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:45.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:45 smithi122 ceph-mon[36397]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:45 smithi145 ceph-mon[36730]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:45.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.788+0000 7effe0400640 1 -- 172.21.15.107:0/3355101111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 msgr2=0x7effd8075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:45.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.788+0000 7effe0400640 1 --2- 172.21.15.107:0/3355101111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd8075f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7effcc009900 tx=0x7effcc02bc50 comp rx=0 tx=0).stop 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 -- 172.21.15.107:0/3355101111 shutdown_connections 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 --2- 172.21.15.107:0/3355101111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8077720 0x7effd8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 --2- 172.21.15.107:0/3355101111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8076d80 0x7effd80771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 --2- 172.21.15.107:0/3355101111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd8075f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 -- 172.21.15.107:0/3355101111 >> 172.21.15.107:0/3355101111 conn(0x7effd80fe970 msgr2=0x7effd8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 -- 172.21.15.107:0/3355101111 shutdown_connections 2024-03-11T19:49:45.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.789+0000 7effe0400640 1 -- 172.21.15.107:0/3355101111 wait complete. 2024-03-11T19:49:45.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 Processor -- start 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 -- start start 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8076d80 0x7effd819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8077720 0x7effd81a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a5330 con 0x7effd8077720 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a54a0 con 0x7effd8076d80 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effe0400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81a5660 con 0x7effd8075b40 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effdda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8077720 0x7effd81a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effdd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8076d80 0x7effd819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effdda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51574/0 (socket says 172.21.15.107:51574) 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.790+0000 7effdda00640 1 -- 172.21.15.107:0/4094832716 learned_addr learned my addr 172.21.15.107:0/4094832716 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 -- 172.21.15.107:0/4094832716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8076d80 msgr2=0x7effd819dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8076d80 0x7effd819dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:45.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 -- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8077720 msgr2=0x7effd81a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8077720 0x7effd81a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 -- 172.21.15.107:0/4094832716 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0009840 con 0x7effd8075b40 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effdda00640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effcc02c6c0 tx=0x7effcc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0044e0 con 0x7effd8075b40 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc009560 con 0x7effd8075b40 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effcc004ac0 con 0x7effd8075b40 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.791+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc003e80 con 0x7effd8075b40 2024-03-11T19:49:45.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.792+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd81a5c60 con 0x7effd8075b40 2024-03-11T19:49:45.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.793+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd810fe60 con 0x7effd8075b40 2024-03-11T19:49:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.795+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7effcc0035f0 con 0x7effd8075b40 2024-03-11T19:49:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.796+0000 7effd6000640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 0x7effc00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.796+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7effcc0ef930 con 0x7effd8075b40 2024-03-11T19:49:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.796+0000 7effdd000640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 0x7effc00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.797+0000 7effdd000640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 0x7effc00a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd819ed50 tx=0x7effc400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:45.800+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7effcc0b8f40 con 0x7effd8075b40 2024-03-11T19:49:46.042 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:45 smithi107 systemd[1]: Starting Ceph mds.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:46.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.044+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effd81131d0 con 0x7effd8075b40 2024-03-11T19:49:46.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.045+0000 7effd6000640 1 -- 172.21.15.107:0/4094832716 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7effcc0bc600 con 0x7effd8075b40 2024-03-11T19:49:46.050 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:46.050 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:49:46.050 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 msgr2=0x7effc00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 0x7effc00a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd819ed50 tx=0x7effc400b410 comp rx=0 tx=0).stop 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 msgr2=0x7effd819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7effcc02c6c0 tx=0x7effcc0042f0 comp rx=0 tx=0).stop 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 shutdown_connections 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7effc00a1a30 0x7effc00a3ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effd8076d80 0x7effd819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:46.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd8077720 0x7effd81a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:46.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 --2- 172.21.15.107:0/4094832716 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7effd8075b40 0x7effd819d840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:46.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 >> 172.21.15.107:0/4094832716 conn(0x7effd80fe970 msgr2=0x7effd8100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:46.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 shutdown_connections 2024-03-11T19:49:46.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:46.048+0000 7effe0400640 1 -- 172.21.15.107:0/4094832716 wait complete. 2024-03-11T19:49:46.295 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 podman[73109]: 2024-03-11 19:49:45.943710176 +0000 UTC m=+0.034715106 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:46.295 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 podman[73109]: 2024-03-11 19:49:46.083460816 +0000 UTC m=+0.174465763 container create 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-11T19:49:46.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:46 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4094832716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:46.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:46 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4094832716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:46 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4094832716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 podman[73109]: 2024-03-11 19:49:46.396412809 +0000 UTC m=+0.487417783 container init 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 podman[73109]: 2024-03-11 19:49:46.406652632 +0000 UTC m=+0.497657601 container start 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 ceph-mds[73130]: -- 172.21.15.107:0/618762374 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b095901a0 con 0x561b09566800 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: starting mds.a at 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 bash[73109]: 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 2024-03-11T19:49:46.795 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 19:49:46 smithi107 systemd[1]: Started Ceph mds.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:47.625 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:47.625 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.625 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.625 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.626 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:47.626 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:47.626 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:47 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:47.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:47.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:47 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:49:47.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:47 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:48.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:48.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.291+0000 7fe535000640 1 -- 172.21.15.107:0/125881099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530077720 msgr2=0x7fe530077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.291+0000 7fe535000640 1 --2- 172.21.15.107:0/125881099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530077720 0x7fe530077fc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe528009ca0 tx=0x7fe52802c100 comp rx=0 tx=0).stop 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 -- 172.21.15.107:0/125881099 shutdown_connections 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 --2- 172.21.15.107:0/125881099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530077720 0x7fe530077fc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 --2- 172.21.15.107:0/125881099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530076d80 0x7fe5300771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 --2- 172.21.15.107:0/125881099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 0x7fe530075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 -- 172.21.15.107:0/125881099 >> 172.21.15.107:0/125881099 conn(0x7fe5300fe930 msgr2=0x7fe530100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 -- 172.21.15.107:0/125881099 shutdown_connections 2024-03-11T19:49:48.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.292+0000 7fe535000640 1 -- 172.21.15.107:0/125881099 wait complete. 2024-03-11T19:49:48.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 Processor -- start 2024-03-11T19:49:48.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 -- start start 2024-03-11T19:49:48.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 0x7fe53019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:48.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:48.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530077720 0x7fe5301a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301a5300 con 0x7fe530075b40 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301a5470 con 0x7fe530076d80 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe535000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5301a5630 con 0x7fe530077720 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe52de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 0x7fe53019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe527e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe527e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38950/0 (socket says 172.21.15.107:38950) 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe527e00640 1 -- 172.21.15.107:0/3072326885 learned_addr learned my addr 172.21.15.107:0/3072326885 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.293+0000 7fe52e800640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530077720 0x7fe5301a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 -- 172.21.15.107:0/3072326885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530077720 msgr2=0x7fe5301a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530077720 0x7fe5301a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 -- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 msgr2=0x7fe53019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 0x7fe53019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 -- 172.21.15.107:0/3072326885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528009900 con 0x7fe530076d80 2024-03-11T19:49:48.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe527e00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9f0 tx=0x7fe51800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518009720 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe51800cd80 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518010690 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5301a5910 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.294+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5301a5dd0 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.296+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53010fec0 con 0x7fe530076d80 2024-03-11T19:49:48.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.297+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe5180046c0 con 0x7fe530076d80 2024-03-11T19:49:48.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.298+0000 7fe526a00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 0x7fe5140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:48.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.299+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe518014070 con 0x7fe530076d80 2024-03-11T19:49:48.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.299+0000 7fe52de00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 0x7fe5140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:48.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.299+0000 7fe52de00640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 0x7fe5140a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe520009bd0 tx=0x7fe520005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:48.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.302+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe518096980 con 0x7fe530076d80 2024-03-11T19:49:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.538+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe53010f930 con 0x7fe530076d80 2024-03-11T19:49:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.539+0000 7fe526a00640 1 -- 172.21.15.107:0/3072326885 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9260 (secure 0 0 0) 0x7fe51809a040 con 0x7fe530076d80 2024-03-11T19:49:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:48.544 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-11T19:49:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-03-11T19:49:48.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:48 smithi107 ceph-mon[31712]: Deploying daemon mds.b on smithi122 2024-03-11T19:49:48.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:48 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:boot 2024-03-11T19:49:48.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:48 smithi107 ceph-mon[31712]: fsmap 11 up:standby 2024-03-11T19:49:48.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:48 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:49:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 msgr2=0x7fe5140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 0x7fe5140a3ef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe520009bd0 tx=0x7fe520005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 msgr2=0x7fe53019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9f0 tx=0x7fe51800edf0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 shutdown_connections 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5140a1a30 0x7fe5140a3ef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe530076d80 0x7fe53019dd50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe530077720 0x7fe5301a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 --2- 172.21.15.107:0/3072326885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe530075b40 0x7fe53019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 >> 172.21.15.107:0/3072326885 conn(0x7fe5300fe930 msgr2=0x7fe530100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:48.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 shutdown_connections 2024-03-11T19:49:48.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:48.542+0000 7fe535000640 1 -- 172.21.15.107:0/3072326885 wait complete. 2024-03-11T19:49:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:48 smithi122 ceph-mon[36397]: Deploying daemon mds.b on smithi122 2024-03-11T19:49:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:48 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:boot 2024-03-11T19:49:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:48 smithi122 ceph-mon[36397]: fsmap 11 up:standby 2024-03-11T19:49:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:48 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:49:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:48 smithi145 ceph-mon[36730]: Deploying daemon mds.b on smithi122 2024-03-11T19:49:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:48 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:boot 2024-03-11T19:49:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:48 smithi145 ceph-mon[36730]: fsmap 11 up:standby 2024-03-11T19:49:48.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:48 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:49:49.212 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:48 smithi122 systemd[1]: Starting Ceph mds.b for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T19:49:49.462 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:49 smithi122 ceph-mon[36397]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:49.463 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:49 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3072326885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:49.463 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 podman[66928]: 2024-03-11 19:49:49.111652192 +0000 UTC m=+0.018073836 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:49:49.463 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 podman[66928]: 2024-03-11 19:49:49.231139631 +0000 UTC m=+0.137561303 container create 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 podman[66928]: 2024-03-11 19:49:49.459016663 +0000 UTC m=+0.365438300 container init 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 podman[66928]: 2024-03-11 19:49:49.461787806 +0000 UTC m=+0.368209436 container start 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 ceph-mds[66947]: -- 172.21.15.122:0/3675277070 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d7d6f5e1a0 con 0x55d7d6f35000 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: starting mds.b at 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 bash[66928]: 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 2024-03-11T19:49:49.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:49:49 smithi122 systemd[1]: Started Ceph mds.b for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T19:49:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:49 smithi145 ceph-mon[36730]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:49 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3072326885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:49 smithi107 ceph-mon[31712]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:49.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:49 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3072326885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:50.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:50.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 -- 172.21.15.107:0/3360777571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 msgr2=0x7f4d2c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:50.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 --2- 172.21.15.107:0/3360777571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c102810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:50.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 -- 172.21.15.107:0/3360777571 shutdown_connections 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 --2- 172.21.15.107:0/3360777571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c102810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 --2- 172.21.15.107:0/3360777571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 --2- 172.21.15.107:0/3360777571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 -- 172.21.15.107:0/3360777571 >> 172.21.15.107:0/3360777571 conn(0x7f4d2c0fc730 msgr2=0x7f4d2c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 -- 172.21.15.107:0/3360777571 shutdown_connections 2024-03-11T19:49:50.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.750+0000 7f4d34400640 1 -- 172.21.15.107:0/3360777571 wait complete. 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.751+0000 7f4d34400640 1 Processor -- start 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 -- start start 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1a0ef0 con 0x7f4d2c102d50 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1a1060 con 0x7f4d2c102390 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d34400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1a1220 con 0x7f4d2c1011a0 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d32400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51626/0 (socket says 172.21.15.107:51626) 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31a00640 1 -- 172.21.15.107:0/679578894 learned_addr learned my addr 172.21.15.107:0/679578894 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31000640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31a00640 1 -- 172.21.15.107:0/679578894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 msgr2=0x7f4d2c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.752+0000 7f4d31a00640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d31a00640 1 -- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 msgr2=0x7f4d2c1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d31a00640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d31a00640 1 -- 172.21.15.107:0/679578894 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009560 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d31000640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d31a00640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e9c0 tx=0x7f4d2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d200096e0 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c1a1500 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d2000cd80 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20010430 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.753+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c1a19e0 con 0x7f4d2c1011a0 2024-03-11T19:49:50.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.754+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c10cc00 con 0x7f4d2c1011a0 2024-03-11T19:49:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.756+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4d20003680 con 0x7f4d2c1011a0 2024-03-11T19:49:50.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.757+0000 7f4d2a000640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 0x7f4d140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:50.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.758+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d20014070 con 0x7f4d2c1011a0 2024-03-11T19:49:50.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.758+0000 7f4d31000640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 0x7f4d140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:50.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.758+0000 7f4d31000640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 0x7f4d140a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d1802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:50.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:50.764+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d20096980 con 0x7f4d2c1011a0 2024-03-11T19:49:50.903 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:50.903 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:boot 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: fsmap 12 up:standby 2024-03-11T19:49:50.904 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:50 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:51.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:51.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:boot 2024-03-11T19:49:51.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: fsmap 12 up:standby 2024-03-11T19:49:51.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:50 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:49:51.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.019+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d2c10cd90 con 0x7f4d2c1011a0 2024-03-11T19:49:51.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.020+0000 7f4d2a000640 1 -- 172.21.15.107:0/679578894 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10055 (secure 0 0 0) 0x7f4d2009a040 con 0x7f4d2c1011a0 2024-03-11T19:49:51.024 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:51.024 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-11T19:49:51.024 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 msgr2=0x7f4d140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 0x7f4d140a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000c00 tx=0x7f4d1802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 msgr2=0x7f4d2c199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e9c0 tx=0x7f4d2000ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 shutdown_connections 2024-03-11T19:49:51.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4d140a1a30 0x7f4d140a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d2c102390 0x7f4d2c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2c102d50 0x7f4d2c1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 --2- 172.21.15.107:0/679578894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d2c1011a0 0x7f4d2c199400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.022+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 >> 172.21.15.107:0/679578894 conn(0x7f4d2c0fc730 msgr2=0x7f4d2c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.023+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 shutdown_connections 2024-03-11T19:49:51.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.023+0000 7f4d34400640 1 -- 172.21.15.107:0/679578894 wait complete. 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:boot 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: fsmap 12 up:standby 2024-03-11T19:49:51.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:50 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:49:51.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:49:51.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 -- 172.21.15.107:0/704848894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 msgr2=0x7f637c109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 --2- 172.21.15.107:0/704848894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c109100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6374009ca0 tx=0x7f637402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 -- 172.21.15.107:0/704848894 shutdown_connections 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 --2- 172.21.15.107:0/704848894 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c109100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 --2- 172.21.15.107:0/704848894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c104380 0x7f637c106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 --2- 172.21.15.107:0/704848894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c101a50 0x7f637c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 -- 172.21.15.107:0/704848894 >> 172.21.15.107:0/704848894 conn(0x7f637c0fb460 msgr2=0x7f637c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 -- 172.21.15.107:0/704848894 shutdown_connections 2024-03-11T19:49:51.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.253+0000 7f6380a00640 1 -- 172.21.15.107:0/704848894 wait complete. 2024-03-11T19:49:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.254+0000 7f6380a00640 1 Processor -- start 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.254+0000 7f6380a00640 1 -- start start 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 0x7f637c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a0d10 con 0x7f637c101a50 2024-03-11T19:49:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a0e80 con 0x7f637c104380 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6380a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a1040 con 0x7f637c106cb0 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6379e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 0x7f637c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55074/0 (socket says 172.21.15.107:55074) 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 -- 172.21.15.107:0/249217390 learned_addr learned my addr 172.21.15.107:0/249217390 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:51.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637b200640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 -- 172.21.15.107:0/249217390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 msgr2=0x7f637c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 -- 172.21.15.107:0/249217390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 msgr2=0x7f637c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 0x7f637c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f637a800640 1 -- 172.21.15.107:0/249217390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009490 con 0x7f637c101a50 2024-03-11T19:49:51.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.255+0000 7f6379e00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 0x7f637c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f637a800640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9f0 tx=0x7f636c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c009700 con 0x7f637c101a50 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6374009900 con 0x7f637c101a50 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f636c00cd80 con 0x7f637c101a50 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c018920 con 0x7f637c101a50 2024-03-11T19:49:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.256+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c1a16f0 con 0x7f637c101a50 2024-03-11T19:49:51.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.257+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c067e90 con 0x7f637c101a50 2024-03-11T19:49:51.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.261+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f636c018a80 con 0x7f637c101a50 2024-03-11T19:49:51.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.262+0000 7f6373400640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 0x7f63600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.262+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f636c014070 con 0x7f637c101a50 2024-03-11T19:49:51.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.262+0000 7f6379e00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 0x7f63600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.263+0000 7f6379e00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 0x7f63600a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f637c19a730 tx=0x7f6368005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:51.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.267+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f636c052d20 con 0x7f637c101a50 2024-03-11T19:49:51.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.485+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f637c1087a0 con 0x7f637c101a50 2024-03-11T19:49:51.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.488+0000 7f6373400640 1 -- 172.21.15.107:0/249217390 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18548 (secure 0 0 0) 0x7f636c096a50 con 0x7f637c101a50 2024-03-11T19:49:51.492 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:51.492 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:48:58.522066+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:49:51.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.490+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 msgr2=0x7f63600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.490+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 0x7f63600a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f637c19a730 tx=0x7f6368005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.490+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 msgr2=0x7f637c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.490+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9f0 tx=0x7f636c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 shutdown_connections 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63600a1a30 0x7f63600a3ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f637c104380 0x7f637c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f637c106cb0 0x7f637c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 --2- 172.21.15.107:0/249217390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f637c101a50 0x7f637c199220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 >> 172.21.15.107:0/249217390 conn(0x7f637c0fb460 msgr2=0x7f637c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 shutdown_connections 2024-03-11T19:49:51.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.491+0000 7f6380a00640 1 -- 172.21.15.107:0/249217390 wait complete. 2024-03-11T19:49:51.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:49:51.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:49:51.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 -- 172.21.15.107:0/2727181400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b58104f70 msgr2=0x7f4b5810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2727181400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b58104f70 0x7f4b5810b800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009ca0 tx=0x7f4b5002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 -- 172.21.15.107:0/2727181400 shutdown_connections 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2727181400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b58104f70 0x7f4b5810b800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2727181400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581045b0 0x7f4b58104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2727181400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b581033c0 0x7f4b581037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.724+0000 7f4b5d400640 1 -- 172.21.15.107:0/2727181400 >> 172.21.15.107:0/2727181400 conn(0x7f4b580fe930 msgr2=0x7f4b58100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.725+0000 7f4b5d400640 1 -- 172.21.15.107:0/2727181400 shutdown_connections 2024-03-11T19:49:51.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.725+0000 7f4b5d400640 1 -- 172.21.15.107:0/2727181400 wait complete. 2024-03-11T19:49:51.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 Processor -- start 2024-03-11T19:49:51.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 -- start start 2024-03-11T19:49:51.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 0x7f4b58199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b58104f70 0x7f4b581a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581a0ed0 con 0x7f4b581033c0 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581a1040 con 0x7f4b581045b0 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b5d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581a1200 con 0x7f4b58104f70 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b56800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b55e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 0x7f4b58199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b56800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55098/0 (socket says 172.21.15.107:55098) 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.726+0000 7f4b56800640 1 -- 172.21.15.107:0/2452564918 learned_addr learned my addr 172.21.15.107:0/2452564918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b57200640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b58104f70 0x7f4b581a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 -- 172.21.15.107:0/2452564918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b58104f70 msgr2=0x7f4b581a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b58104f70 0x7f4b581a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 -- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 msgr2=0x7f4b581993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 -- 172.21.15.107:0/2452564918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009490 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b56800640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b55e00640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 0x7f4b58199920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d6d0 tx=0x7f4b4400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.727+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44005220 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.728+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b44009d00 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.728+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b50009900 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.728+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004dd0 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.728+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b581a18b0 con 0x7f4b581045b0 2024-03-11T19:49:51.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.729+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5810a680 con 0x7f4b581045b0 2024-03-11T19:49:51.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.731+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4b44010670 con 0x7f4b581045b0 2024-03-11T19:49:51.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.732+0000 7f4b4f400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 0x7f4b3c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:51.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.732+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4b440cca00 con 0x7f4b581045b0 2024-03-11T19:49:51.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.733+0000 7f4b56800640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 0x7f4b3c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:51.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.733+0000 7f4b56800640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 0x7f4b3c0a3f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b480058c0 tx=0x7f4b48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:51.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.736+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b44096100 con 0x7f4b581045b0 2024-03-11T19:49:51.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.970+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b5802cda0 con 0x7f4b581045b0 2024-03-11T19:49:51.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.971+0000 7f4b4f400640 1 -- 172.21.15.107:0/2452564918 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b440997c0 con 0x7f4b581045b0 2024-03-11T19:49:51.975 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:49:51.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 msgr2=0x7f4b3c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 0x7f4b3c0a3f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b480058c0 tx=0x7f4b48005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:51.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 msgr2=0x7f4b58199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 0x7f4b58199920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d6d0 tx=0x7f4b4400db90 comp rx=0 tx=0).stop 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 shutdown_connections 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b3c0a1ab0 0x7f4b3c0a3f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b581045b0 0x7f4b58199920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4b58104f70 0x7f4b581a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 --2- 172.21.15.107:0/2452564918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b581033c0 0x7f4b581993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.973+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 >> 172.21.15.107:0/2452564918 conn(0x7f4b580fe930 msgr2=0x7f4b58107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.974+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 shutdown_connections 2024-03-11T19:49:51.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:51.974+0000 7f4b5d400640 1 -- 172.21.15.107:0/2452564918 wait complete. 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/679578894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/249217390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:51 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:51.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-11T19:49:52.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/679578894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:52.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/249217390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:52.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:51 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/679578894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:52.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/249217390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:49:52.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:51 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 -- 172.21.15.107:0/3267641747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc101190 msgr2=0x7ff7fc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 --2- 172.21.15.107:0/3267641747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc101190 0x7ff7fc1015b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009900 tx=0x7ff7f002bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 -- 172.21.15.107:0/3267641747 shutdown_connections 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 --2- 172.21.15.107:0/3267641747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc102380 0x7ff7fc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 --2- 172.21.15.107:0/3267641747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 0x7ff7fc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 --2- 172.21.15.107:0/3267641747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc101190 0x7ff7fc1015b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 -- 172.21.15.107:0/3267641747 >> 172.21.15.107:0/3267641747 conn(0x7ff7fc0fc720 msgr2=0x7ff7fc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.205+0000 7ff804000640 1 -- 172.21.15.107:0/3267641747 shutdown_connections 2024-03-11T19:49:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.206+0000 7ff804000640 1 -- 172.21.15.107:0/3267641747 wait complete. 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.206+0000 7ff804000640 1 Processor -- start 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 -- start start 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 0x7ff7fc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 0x7ff7fc1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a0ee0 con 0x7ff7fc102380 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a1050 con 0x7ff7fc101190 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff804000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a1210 con 0x7ff7fc102d40 2024-03-11T19:49:52.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff800c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 0x7ff7fc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff801600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff801600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39026/0 (socket says 172.21.15.107:39026) 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff800c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 0x7ff7fc199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55120/0 (socket says 172.21.15.107:55120) 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff801600640 1 -- 172.21.15.107:0/1703083994 learned_addr learned my addr 172.21.15.107:0/1703083994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.207+0000 7ff802000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 0x7ff7fc1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 -- 172.21.15.107:0/1703083994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 msgr2=0x7ff7fc1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 0x7ff7fc1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 -- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 msgr2=0x7ff7fc199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 0x7ff7fc199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 -- 172.21.15.107:0/1703083994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009560 con 0x7ff7fc101190 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff801600640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00044c0 con 0x7ff7fc101190 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f00052d0 con 0x7ff7fc101190 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f003e450 con 0x7ff7fc101190 2024-03-11T19:49:52.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.208+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc1a1490 con 0x7ff7fc101190 2024-03-11T19:49:52.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.209+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc1a1950 con 0x7ff7fc101190 2024-03-11T19:49:52.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.210+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc10cbf0 con 0x7ff7fc101190 2024-03-11T19:49:52.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.211+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff7f0005440 con 0x7ff7fc101190 2024-03-11T19:49:52.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.213+0000 7ff7fa000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 0x7ff7e40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.213+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7f00ef7b0 con 0x7ff7fc101190 2024-03-11T19:49:52.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.213+0000 7ff800c00640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 0x7ff7e40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.214+0000 7ff800c00640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 0x7ff7e40a3f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc19a900 tx=0x7ff7e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:52.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.217+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7f00b8ee0 con 0x7ff7fc101190 2024-03-11T19:49:52.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.431+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff7fc10cdd0 con 0x7ff7fc101190 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.432+0000 7ff7fa000640 1 -- 172.21.15.107:0/1703083994 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff7f00bc5a0 con 0x7ff7fc101190 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T19:49:52.437 INFO:teuthology.orchestra.run.smithi107.stdout:] 2024-03-11T19:49:52.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.434+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 msgr2=0x7ff7e40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.434+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 0x7ff7e40a3f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc19a900 tx=0x7ff7e800b410 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.434+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 msgr2=0x7ff7fc1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.434+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000c00 tx=0x7ff7f00040c0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 shutdown_connections 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff7e40a1ab0 0x7ff7e40a3f70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff7fc102d40 0x7ff7fc1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7fc102380 0x7ff7fc199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 --2- 172.21.15.107:0/1703083994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7fc101190 0x7ff7fc1993f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 >> 172.21.15.107:0/1703083994 conn(0x7ff7fc0fc720 msgr2=0x7ff7fc0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 shutdown_connections 2024-03-11T19:49:52.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.435+0000 7ff804000640 1 -- 172.21.15.107:0/1703083994 wait complete. 2024-03-11T19:49:52.458 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-11T19:49:52.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 -- 172.21.15.107:0/3061077408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81045b0 msgr2=0x7fa8c8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3061077408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81045b0 0x7fa8c8104a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009900 tx=0x7fa8b402bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 -- 172.21.15.107:0/3061077408 shutdown_connections 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3061077408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c8104f70 0x7fa8c810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3061077408 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81045b0 0x7fa8c8104a30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3061077408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c81033c0 0x7fa8c81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 -- 172.21.15.107:0/3061077408 >> 172.21.15.107:0/3061077408 conn(0x7fa8c80fe930 msgr2=0x7fa8c8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:52.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 -- 172.21.15.107:0/3061077408 shutdown_connections 2024-03-11T19:49:52.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.663+0000 7fa8ce200640 1 -- 172.21.15.107:0/3061077408 wait complete. 2024-03-11T19:49:52.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.664+0000 7fa8ce200640 1 Processor -- start 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.664+0000 7fa8ce200640 1 -- start start 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c81045b0 0x7fa8c8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c8104f70 0x7fa8c81a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a0f30 con 0x7fa8c8104f70 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a10a0 con 0x7fa8c81045b0 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8ce200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81a1260 con 0x7fa8c81033c0 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51704/0 (socket says 172.21.15.107:51704) 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 -- 172.21.15.107:0/3440580728 learned_addr learned my addr 172.21.15.107:0/3440580728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c6a00640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c81045b0 0x7fa8c8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7e00640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c8104f70 0x7fa8c81a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 -- 172.21.15.107:0/3440580728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c81045b0 msgr2=0x7fa8c8199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c81045b0 0x7fa8c8199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 -- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c8104f70 msgr2=0x7fa8c81a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c8104f70 0x7fa8c81a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.665+0000 7fa8c7400640 1 -- 172.21.15.107:0/3440580728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4009560 con 0x7fa8c81033c0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8c7400640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9c0 tx=0x7fa8b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8009700 con 0x7fa8c81033c0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b800cd80 con 0x7fa8c81033c0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8010430 con 0x7fa8c81033c0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c81a1540 con 0x7fa8c81033c0 2024-03-11T19:49:52.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.666+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81a19d0 con 0x7fa8c81033c0 2024-03-11T19:49:52.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.668+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c810a680 con 0x7fa8c81033c0 2024-03-11T19:49:52.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.673+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa8b8003680 con 0x7fa8c81033c0 2024-03-11T19:49:52.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.674+0000 7fa8bfe00640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 0x7fa8ac0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:52.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.674+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8b8014070 con 0x7fa8c81033c0 2024-03-11T19:49:52.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.675+0000 7fa8c6a00640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 0x7fa8ac0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:52.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.675+0000 7fa8c6a00640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 0x7fa8ac0a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000c00 tx=0x7fa8b402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:52.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.677+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8b8096b50 con 0x7fa8c81033c0 2024-03-11T19:49:52.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2452564918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1703083994' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:52.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:52 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:52.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:52.882+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa8c810efa0 con 0x7fa8c81033c0 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2452564918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1703083994' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:52 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:53.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2452564918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:49:53.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:53.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1703083994' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:49:53.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:53.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:53.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:53.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:52 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:53.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.697+0000 7fa8bfe00640 1 -- 172.21.15.107:0/3440580728 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa8b809a210 con 0x7fa8c81033c0 2024-03-11T19:49:53.706 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.703+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 msgr2=0x7fa8ac0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.703+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 0x7fa8ac0a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000c00 tx=0x7fa8b402c7e0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.703+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 msgr2=0x7fa8c8199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.703+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9c0 tx=0x7fa8b800ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 shutdown_connections 2024-03-11T19:49:53.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa8ac0a1a30 0x7fa8ac0a3ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c81045b0 0x7fa8c8199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8c8104f70 0x7fa8c81a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 --2- 172.21.15.107:0/3440580728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8c81033c0 0x7fa8c8199440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 >> 172.21.15.107:0/3440580728 conn(0x7fa8c80fe930 msgr2=0x7fa8c8107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 shutdown_connections 2024-03-11T19:49:53.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.704+0000 7fa8ce200640 1 -- 172.21.15.107:0/3440580728 wait complete. 2024-03-11T19:49:53.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-11T19:49:53.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 -- 172.21.15.107:0/2774145919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c1000a0 msgr2=0x7f121c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 --2- 172.21.15.107:0/2774145919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c1000a0 0x7f121c1004c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f120c009860 tx=0x7f120c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 -- 172.21.15.107:0/2774145919 shutdown_connections 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 --2- 172.21.15.107:0/2774145919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101c50 0x7f121c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 --2- 172.21.15.107:0/2774145919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c101290 0x7f121c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 --2- 172.21.15.107:0/2774145919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c1000a0 0x7f121c1004c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 -- 172.21.15.107:0/2774145919 >> 172.21.15.107:0/2774145919 conn(0x7f121c0fb630 msgr2=0x7f121c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.921+0000 7f1222c00640 1 -- 172.21.15.107:0/2774145919 shutdown_connections 2024-03-11T19:49:53.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.922+0000 7f1222c00640 1 -- 172.21.15.107:0/2774145919 wait complete. 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 Processor -- start 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 -- start start 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c1000a0 0x7f121c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101290 0x7f121c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c1a0fa0 con 0x7f121c101290 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c1a1110 con 0x7f121c101c50 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1222c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c1a12d0 con 0x7f121c1000a0 2024-03-11T19:49:53.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1220c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f121be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c1000a0 0x7f121c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1220c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39040/0 (socket says 172.21.15.107:39040) 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f121b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101290 0x7f121c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.923+0000 7f1220c00640 1 -- 172.21.15.107:0/1305566378 learned_addr learned my addr 172.21.15.107:0/1305566378 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 -- 172.21.15.107:0/1305566378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c1000a0 msgr2=0x7f121c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c1000a0 0x7f121c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 -- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101290 msgr2=0x7f121c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101290 0x7f121c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 -- 172.21.15.107:0/1305566378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009560 con 0x7f121c101c50 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1220c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f121400eaa0 tx=0x7f121400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400cdb0 con 0x7f121c101c50 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c009490 con 0x7f121c101c50 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1214005020 con 0x7f121c101c50 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.924+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140116b0 con 0x7f121c101c50 2024-03-11T19:49:53.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.925+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c1a18e0 con 0x7f121c101c50 2024-03-11T19:49:53.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.926+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c10cbe0 con 0x7f121c101c50 2024-03-11T19:49:53.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.927+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1214004b20 con 0x7f121c101c50 2024-03-11T19:49:53.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.928+0000 7f1218c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 0x7f12000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:53.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.929+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f12140cd380 con 0x7f121c101c50 2024-03-11T19:49:53.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.929+0000 7f121be00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 0x7f12000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:53.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.929+0000 7f121be00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 0x7f12000a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f120c009830 tx=0x7f120c034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:53.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:53.933+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12140968d0 con 0x7f121c101c50 2024-03-11T19:49:54.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:54.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:53 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3440580728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:53 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:53 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:54.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:54.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:53 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3440580728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:53 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:53 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:54.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:54.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:53 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3440580728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:53 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:49:54.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:53 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.146+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f121c1a1de0 con 0x7f121c101c50 2024-03-11T19:49:54.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.697+0000 7f1218c00640 1 -- 172.21.15.107:0/1305566378 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f1214099f90 con 0x7f121c101c50 2024-03-11T19:49:54.703 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2024-03-11T19:49:54.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.706+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 msgr2=0x7f12000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:54.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.706+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 0x7f12000a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f120c009830 tx=0x7f120c034c00 comp rx=0 tx=0).stop 2024-03-11T19:49:54.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.706+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 msgr2=0x7f121c1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:54.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.706+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f121400eaa0 tx=0x7f121400ef60 comp rx=0 tx=0).stop 2024-03-11T19:49:54.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 shutdown_connections 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f12000a1a30 0x7f12000a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f121c101c50 0x7f121c1a0a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f121c101290 0x7f121c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 --2- 172.21.15.107:0/1305566378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f121c1000a0 0x7f121c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.707+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 >> 172.21.15.107:0/1305566378 conn(0x7f121c0fb630 msgr2=0x7f121c10a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.708+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 shutdown_connections 2024-03-11T19:49:54.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.708+0000 7f1222c00640 1 -- 172.21.15.107:0/1305566378 wait complete. 2024-03-11T19:49:54.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-11T19:49:54.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 -- 172.21.15.107:0/200456993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c104470 msgr2=0x7f9f4c1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 --2- 172.21.15.107:0/200456993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c104470 0x7f9f4c1048f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009900 tx=0x7f9f3802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 -- 172.21.15.107:0/200456993 shutdown_connections 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 --2- 172.21.15.107:0/200456993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c104470 0x7f9f4c1048f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 --2- 172.21.15.107:0/200456993 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104e30 0x7f9f4c10b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 --2- 172.21.15.107:0/200456993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c103280 0x7f9f4c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 -- 172.21.15.107:0/200456993 >> 172.21.15.107:0/200456993 conn(0x7f9f4c0fe7f0 msgr2=0x7f9f4c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 -- 172.21.15.107:0/200456993 shutdown_connections 2024-03-11T19:49:54.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.913+0000 7f9f4be00640 1 -- 172.21.15.107:0/200456993 wait complete. 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.914+0000 7f9f4be00640 1 Processor -- start 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.914+0000 7f9f4be00640 1 -- start start 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 0x7f9f4c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 0x7f9f4c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:54.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c1a0e60 con 0x7f9f4c104e30 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c1a0fd0 con 0x7f9f4c103280 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c1a1190 con 0x7f9f4c104470 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 0x7f9f4c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39058/0 (socket says 172.21.15.107:39058) 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 -- 172.21.15.107:0/4255377625 learned_addr learned my addr 172.21.15.107:0/4255377625 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4a000640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 0x7f9f4c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 -- 172.21.15.107:0/4255377625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 msgr2=0x7f9f4c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 0x7f9f4c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 -- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 msgr2=0x7f9f4c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 0x7f9f4c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.915+0000 7f9f4aa00640 1 -- 172.21.15.107:0/4255377625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009560 con 0x7f9f4c103280 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f4a000640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 0x7f9f4c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f4b400640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 0x7f9f4c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f4aa00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9c0 tx=0x7f9f3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c009700 con 0x7f9f4c103280 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c1a1470 con 0x7f9f4c103280 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c00cd80 con 0x7f9f4c103280 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c010430 con 0x7f9f4c103280 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.916+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c1a1950 con 0x7f9f4c103280 2024-03-11T19:49:54.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.917+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c10a540 con 0x7f9f4c103280 2024-03-11T19:49:54.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.919+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9f3c003680 con 0x7f9f4c103280 2024-03-11T19:49:54.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.920+0000 7f9f43400640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 0x7f9f300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:54.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.921+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9f3c014070 con 0x7f9f4c103280 2024-03-11T19:49:54.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.921+0000 7f9f4a000640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 0x7f9f300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:54.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.921+0000 7f9f4a000640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 0x7f9f300a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000c00 tx=0x7f9f3802c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:54.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:54.923+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f3c0969b0 con 0x7f9f4c103280 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1305566378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: pgmap v312: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:55.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:49:55.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:49:55.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1305566378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: pgmap v312: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:55.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:55.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:49:55.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:49:55.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1305566378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-11T19:49:55.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: pgmap v312: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:55.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:55.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:55.173+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9f4c10ece0 con 0x7f9f4c103280 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4255377625' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:49:56.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:55 smithi122 ceph-mon[36397]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:49:56.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:49:56.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:49:56.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:49:56.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4255377625' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:55 smithi145 ceph-mon[36730]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4255377625' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:49:56.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:49:56.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:56.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:55 smithi107 ceph-mon[31712]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:49:56.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.174+0000 7f9f43400640 1 -- 172.21.15.107:0/4255377625 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9f3c09a070 con 0x7f9f4c103280 2024-03-11T19:49:56.179 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2024-03-11T19:49:56.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.178+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 msgr2=0x7f9f300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:56.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.178+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 0x7f9f300a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000c00 tx=0x7f9f3802c740 comp rx=0 tx=0).stop 2024-03-11T19:49:56.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 msgr2=0x7f9f4c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e9c0 tx=0x7f9f3c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 shutdown_connections 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9f300a1a30 0x7f9f300a3ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f4c104e30 0x7f9f4c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f4c104470 0x7f9f4c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 --2- 172.21.15.107:0/4255377625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f4c103280 0x7f9f4c199370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.179+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 >> 172.21.15.107:0/4255377625 conn(0x7f9f4c0fe7f0 msgr2=0x7f9f4c107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:56.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.181+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 shutdown_connections 2024-03-11T19:49:56.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.182+0000 7f9f4be00640 1 -- 172.21.15.107:0/4255377625 wait complete. 2024-03-11T19:49:56.211 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-11T19:49:56.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.391+0000 7f4de7200640 1 -- 172.21.15.107:0/76826367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8103280 msgr2=0x7f4de81036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.391+0000 7f4de7200640 1 --2- 172.21.15.107:0/76826367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8103280 0x7f4de81036a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009900 tx=0x7f4dd802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 -- 172.21.15.107:0/76826367 shutdown_connections 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 --2- 172.21.15.107:0/76826367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8104e30 0x7f4de810b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 --2- 172.21.15.107:0/76826367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 0x7f4de81048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 --2- 172.21.15.107:0/76826367 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8103280 0x7f4de81036a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 -- 172.21.15.107:0/76826367 >> 172.21.15.107:0/76826367 conn(0x7f4de80fe7f0 msgr2=0x7f4de8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 -- 172.21.15.107:0/76826367 shutdown_connections 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.392+0000 7f4de7200640 1 -- 172.21.15.107:0/76826367 wait complete. 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 Processor -- start 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 -- start start 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 0x7f4de8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8104e30 0x7f4de81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a0dd0 con 0x7f4de8104470 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a0f40 con 0x7f4de8103280 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.393+0000 7f4de7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81a1100 con 0x7f4de8104e30 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47898/0 (socket says 172.21.15.107:47898) 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 -- 172.21.15.107:0/550780026 learned_addr learned my addr 172.21.15.107:0/550780026 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5400640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 0x7f4de8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de6800640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8104e30 0x7f4de81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:56.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 -- 172.21.15.107:0/550780026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8104e30 msgr2=0x7f4de81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8104e30 0x7f4de81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 -- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 msgr2=0x7f4de8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 0x7f4de8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 -- 172.21.15.107:0/550780026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8009560 con 0x7f4de8103280 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.394+0000 7f4de5e00640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005ab0 tx=0x7f4dd8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.395+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80043d0 con 0x7f4de8103280 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.395+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80052d0 con 0x7f4de8103280 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.395+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de81a1380 con 0x7f4de8103280 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.395+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8003d40 con 0x7f4de8103280 2024-03-11T19:49:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.395+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81a1890 con 0x7f4de8103280 2024-03-11T19:49:56.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.396+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de810a5a0 con 0x7f4de8103280 2024-03-11T19:49:56.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.399+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4dd8005440 con 0x7f4de8103280 2024-03-11T19:49:56.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.400+0000 7f4ddea00640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 0x7f4dcc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:56.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.400+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4dd80ef3d0 con 0x7f4de8103280 2024-03-11T19:49:56.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.400+0000 7f4de5400640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 0x7f4dcc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:56.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.401+0000 7f4de5400640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 0x7f4dcc0a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4de819a7f0 tx=0x7f4dd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:56.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.404+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4dd80b8730 con 0x7f4de8103280 2024-03-11T19:49:56.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:56.616+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f4de802cda0 con 0x7f4de8103280 2024-03-11T19:49:57.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.216+0000 7f4ddea00640 1 -- 172.21.15.107:0/550780026 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7f4dd80bbdf0 con 0x7f4de8103280 2024-03-11T19:49:57.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.220+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 msgr2=0x7f4dcc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:57.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.220+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 0x7f4dcc0a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4de819a7f0 tx=0x7f4dd4005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:57.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.220+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 msgr2=0x7f4de81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:57.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.220+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005ab0 tx=0x7f4dd8005e50 comp rx=0 tx=0).stop 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 shutdown_connections 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4dcc0a1a30 0x7f4dcc0a3ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4de8104e30 0x7f4de81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de8104470 0x7f4de8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 --2- 172.21.15.107:0/550780026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4de8103280 0x7f4de81992e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 >> 172.21.15.107:0/550780026 conn(0x7f4de80fe7f0 msgr2=0x7f4de8107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 shutdown_connections 2024-03-11T19:49:57.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.221+0000 7f4de7200640 1 -- 172.21.15.107:0/550780026 wait complete. 2024-03-11T19:49:57.247 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-11T19:49:57.448 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: pgmap v316: 97 pgs: 25 active+clean, 17 creating+peering, 55 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/550780026' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:57 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 -- 172.21.15.107:0/373377676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 msgr2=0x7f9b6c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 --2- 172.21.15.107:0/373377676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c1037e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009900 tx=0x7f9b5c02bc40 comp rx=0 tx=0).stop 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 -- 172.21.15.107:0/373377676 shutdown_connections 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 --2- 172.21.15.107:0/373377676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c104f70 0x7f9b6c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 --2- 172.21.15.107:0/373377676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c1045b0 0x7f9b6c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 --2- 172.21.15.107:0/373377676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c1037e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 -- 172.21.15.107:0/373377676 >> 172.21.15.107:0/373377676 conn(0x7f9b6c0fe930 msgr2=0x7f9b6c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:57.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 -- 172.21.15.107:0/373377676 shutdown_connections 2024-03-11T19:49:57.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.468+0000 7f9b73000640 1 -- 172.21.15.107:0/373377676 wait complete. 2024-03-11T19:49:57.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.469+0000 7f9b73000640 1 Processor -- start 2024-03-11T19:49:57.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 -- start start 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c1045b0 0x7f9b6c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 0x7f9b6c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1a0f50 con 0x7f9b6c104f70 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1a10c0 con 0x7f9b6c1045b0 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b73000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1a1280 con 0x7f9b6c1033c0 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b6be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b71000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 0x7f9b6c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b6be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50386/0 (socket says 172.21.15.107:50386) 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b71000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 0x7f9b6c1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41304/0 (socket says 172.21.15.107:41304) 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b6be00640 1 -- 172.21.15.107:0/3773757079 learned_addr learned my addr 172.21.15.107:0/3773757079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.470+0000 7f9b6b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c1045b0 0x7f9b6c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 -- 172.21.15.107:0/3773757079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c1045b0 msgr2=0x7f9b6c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c1045b0 0x7f9b6c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 -- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 msgr2=0x7f9b6c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:57.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 0x7f9b6c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:57.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 -- 172.21.15.107:0/3773757079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c009560 con 0x7f9b6c1033c0 2024-03-11T19:49:57.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b6be00640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:57.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0044e0 con 0x7f9b6c1033c0 2024-03-11T19:49:57.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c0054e0 con 0x7f9b6c1033c0 2024-03-11T19:49:57.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c03e450 con 0x7f9b6c1033c0 2024-03-11T19:49:57.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.471+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c1a1500 con 0x7f9b6c1033c0 2024-03-11T19:49:57.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.472+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c1a19c0 con 0x7f9b6c1033c0 2024-03-11T19:49:57.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.473+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c10a680 con 0x7f9b6c1033c0 2024-03-11T19:49:57.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.475+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9b5c0035f0 con 0x7f9b6c1033c0 2024-03-11T19:49:57.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.476+0000 7f9b68c00640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 0x7f9b500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:57.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.476+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f9b5c0f01a0 con 0x7f9b6c1033c0 2024-03-11T19:49:57.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.476+0000 7f9b6b400640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 0x7f9b500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:57.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.477+0000 7f9b6b400640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 0x7f9b500a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c19a970 tx=0x7f9b5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:57.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.478+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b5c0b9320 con 0x7f9b6c1033c0 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: pgmap v316: 97 pgs: 25 active+clean, 17 creating+peering, 55 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/550780026' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:57 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: pgmap v316: 97 pgs: 25 active+clean, 17 creating+peering, 55 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/550780026' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:57 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-11T19:49:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:57.682+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9b6c02ce40 con 0x7f9b6c1033c0 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:active 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: Cluster is now healthy 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3773757079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:58.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:58 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:58.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:active 2024-03-11T19:49:58.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: Cluster is now healthy 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3773757079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:58.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:58 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] up:active 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: Cluster is now healthy 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3773757079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:58.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:58 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:49:59.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.411+0000 7f9b68c00640 1 -- 172.21.15.107:0/3773757079 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7f9b5c043100 con 0x7f9b6c1033c0 2024-03-11T19:49:59.416 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data_ec' created 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.416+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 msgr2=0x7f9b500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.416+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 0x7f9b500a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c19a970 tx=0x7f9b5800b410 comp rx=0 tx=0).stop 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.416+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 msgr2=0x7f9b6c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.416+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c000c00 tx=0x7f9b5c0042f0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 shutdown_connections 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9b500a1a30 0x7f9b500a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b6c1045b0 0x7f9b6c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b6c104f70 0x7f9b6c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 --2- 172.21.15.107:0/3773757079 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b6c1033c0 0x7f9b6c199460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.417+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 >> 172.21.15.107:0/3773757079 conn(0x7f9b6c0fe930 msgr2=0x7f9b6c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:59.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.418+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 shutdown_connections 2024-03-11T19:49:59.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.418+0000 7f9b73000640 1 -- 172.21.15.107:0/3773757079 wait complete. 2024-03-11T19:49:59.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 -- 172.21.15.107:0/3253072695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6581075a0 msgr2=0x7fe6581079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3253072695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6581075a0 0x7fe6581079c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe648009900 tx=0x7fe64802bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 -- 172.21.15.107:0/3253072695 shutdown_connections 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3253072695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658109150 0x7fe65810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3253072695 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 0x7fe658108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3253072695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6581075a0 0x7fe6581079c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 -- 172.21.15.107:0/3253072695 >> 172.21.15.107:0/3253072695 conn(0x7fe658076540 msgr2=0x7fe658078980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 -- 172.21.15.107:0/3253072695 shutdown_connections 2024-03-11T19:49:59.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.635+0000 7fe65dc00640 1 -- 172.21.15.107:0/3253072695 wait complete. 2024-03-11T19:49:59.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 Processor -- start 2024-03-11T19:49:59.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 -- start start 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581075a0 0x7fe65819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 0x7fe65819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581a5140 con 0x7fe658109150 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581a52b0 con 0x7fe6581075a0 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.636+0000 7fe65dc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581a5470 con 0x7fe658108790 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe657e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581075a0 0x7fe65819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41332/0 (socket says 172.21.15.107:41332) 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe64f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 0x7fe65819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 -- 172.21.15.107:0/3307345594 learned_addr learned my addr 172.21.15.107:0/3307345594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 -- 172.21.15.107:0/3307345594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 msgr2=0x7fe65819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 0x7fe65819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 -- 172.21.15.107:0/3307345594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581075a0 msgr2=0x7fe65819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:59.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581075a0 0x7fe65819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 -- 172.21.15.107:0/3307345594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe640009490 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe65d200640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe65000efd0 tx=0x7fe65000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.637+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650009b40 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.638+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe650009840 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.638+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe650003b70 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.638+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6500108f0 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.638+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe648009560 con 0x7fe658109150 2024-03-11T19:49:59.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.639+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65810e8c0 con 0x7fe658109150 2024-03-11T19:49:59.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.641+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe650003ce0 con 0x7fe658109150 2024-03-11T19:49:59.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.642+0000 7fe655600640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 0x7fe63c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:59.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.643+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fe650014070 con 0x7fe658109150 2024-03-11T19:49:59.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.643+0000 7fe657e00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 0x7fe63c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:59.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.643+0000 7fe657e00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 0x7fe63c0a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe648005ab0 tx=0x7fe648005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:59.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.647+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe650096bb0 con 0x7fe658109150 2024-03-11T19:49:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:59 smithi122 ceph-mon[36397]: pgmap v318: 97 pgs: 64 active+clean, 17 creating+peering, 16 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-11T19:49:59.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:49:59 smithi122 ceph-mon[36397]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:49:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:59 smithi145 ceph-mon[36730]: pgmap v318: 97 pgs: 64 active+clean, 17 creating+peering, 16 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-11T19:49:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:49:59 smithi145 ceph-mon[36730]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:49:59.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:59 smithi107 ceph-mon[31712]: pgmap v318: 97 pgs: 64 active+clean, 17 creating+peering, 16 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-03-11T19:49:59.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:49:59 smithi107 ceph-mon[31712]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:49:59.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:49:59.830+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fe658112f80 con 0x7fe658109150 2024-03-11T19:50:00.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.434+0000 7fe655600640 1 -- 172.21.15.107:0/3307345594 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe65009a270 con 0x7fe658109150 2024-03-11T19:50:00.440 INFO:teuthology.orchestra.run.smithi107.stderr:set pool 4 allow_ec_overwrites to true 2024-03-11T19:50:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 msgr2=0x7fe63c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 0x7fe63c0a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe648005ab0 tx=0x7fe648005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 msgr2=0x7fe6581a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe65000efd0 tx=0x7fe65000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 shutdown_connections 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe63c0a1a30 0x7fe63c0a3ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe658109150 0x7fe6581a4c00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe658108790 0x7fe65819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 --2- 172.21.15.107:0/3307345594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581075a0 0x7fe65819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 >> 172.21.15.107:0/3307345594 conn(0x7fe658076540 msgr2=0x7fe65810ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.439+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 shutdown_connections 2024-03-11T19:50:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.440+0000 7fe65dc00640 1 -- 172.21.15.107:0/3307345594 wait complete. 2024-03-11T19:50:00.483 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-11T19:50:00.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 -- 172.21.15.107:0/1802016382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102270 msgr2=0x7fe6601026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 --2- 172.21.15.107:0/1802016382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102270 0x7fe6601026f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe648009900 tx=0x7fe64802bc50 comp rx=0 tx=0).stop 2024-03-11T19:50:00.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 -- 172.21.15.107:0/1802016382 shutdown_connections 2024-03-11T19:50:00.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 --2- 172.21.15.107:0/1802016382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102270 0x7fe6601026f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 --2- 172.21.15.107:0/1802016382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102c30 0x7fe6601094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 --2- 172.21.15.107:0/1802016382 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe6601014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.665+0000 7fe665000640 1 -- 172.21.15.107:0/1802016382 >> 172.21.15.107:0/1802016382 conn(0x7fe6600fc5f0 msgr2=0x7fe6600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.666+0000 7fe665000640 1 -- 172.21.15.107:0/1802016382 shutdown_connections 2024-03-11T19:50:00.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.666+0000 7fe665000640 1 -- 172.21.15.107:0/1802016382 wait complete. 2024-03-11T19:50:00.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.666+0000 7fe665000640 1 Processor -- start 2024-03-11T19:50:00.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 -- start start 2024-03-11T19:50:00.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe660199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102c30 0x7fe6601a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6601a0df0 con 0x7fe660102270 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6601a0f60 con 0x7fe660102c30 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe665000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6601a1120 con 0x7fe660101080 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe65ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe65ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41348/0 (socket says 172.21.15.107:41348) 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe65ea00640 1 -- 172.21.15.107:0/2981150002 learned_addr learned my addr 172.21.15.107:0/2981150002 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe65f400640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe660199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.667+0000 7fe65fe00640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102c30 0x7fe6601a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 -- 172.21.15.107:0/2981150002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 msgr2=0x7fe660199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe660199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 -- 172.21.15.107:0/2981150002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102c30 msgr2=0x7fe6601a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102c30 0x7fe6601a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 -- 172.21.15.107:0/2981150002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650009490 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65f400640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe660199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe65ea00640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6480043d0 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe648009560 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6480052d0 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648003bc0 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.668+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6601a1770 con 0x7fe660102270 2024-03-11T19:50:00.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.669+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66010ca90 con 0x7fe660102270 2024-03-11T19:50:00.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.671+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe648003d20 con 0x7fe660102270 2024-03-11T19:50:00.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.673+0000 7fe657e00640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 0x7fe6440a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.673+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fe6480efd70 con 0x7fe660102270 2024-03-11T19:50:00.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.673+0000 7fe65f400640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 0x7fe6440a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.673+0000 7fe65f400640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 0x7fe6440a3f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6601020d0 tx=0x7fe658009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.675+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6480b8d60 con 0x7fe660102270 2024-03-11T19:50:00.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:50:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: 2024-03-11T19:50:00.396+0000 7fc7e2408700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-11T19:50:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:00 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:50:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:00 smithi122 ceph-mon[36397]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:50:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:00 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:50:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:00 smithi122 ceph-mon[36397]: overall HEALTH_OK 2024-03-11T19:50:00.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:00 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:50:00.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:00 smithi145 ceph-mon[36730]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:50:00.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:00 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:50:00.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:00 smithi145 ceph-mon[36730]: overall HEALTH_OK 2024-03-11T19:50:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:00 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:50:00.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:00 smithi107 ceph-mon[31712]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:50:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:00 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-11T19:50:00.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:00 smithi107 ceph-mon[31712]: overall HEALTH_OK 2024-03-11T19:50:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:00.912+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe66010cc20 con 0x7fe660102270 2024-03-11T19:50:01.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.462+0000 7fe657e00640 1 -- 172.21.15.107:0/2981150002 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe6480bc420 con 0x7fe660102270 2024-03-11T19:50:01.467 INFO:teuthology.orchestra.run.smithi107.stderr:added data pool 4 to fsmap 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 msgr2=0x7fe6440a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 0x7fe6440a3f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe6601020d0 tx=0x7fe658009500 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 msgr2=0x7fe660199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe648006010 tx=0x7fe648005e80 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 shutdown_connections 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe6440a1ab0 0x7fe6440a3f70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe660102c30 0x7fe6601a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe660102270 0x7fe660199840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 --2- 172.21.15.107:0/2981150002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe660101080 0x7fe660199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 >> 172.21.15.107:0/2981150002 conn(0x7fe6600fc5f0 msgr2=0x7fe6600733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 shutdown_connections 2024-03-11T19:50:01.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.465+0000 7fe665000640 1 -- 172.21.15.107:0/2981150002 wait complete. 2024-03-11T19:50:01.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 -- 172.21.15.107:0/4247113598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c501089b0 msgr2=0x7f3c50108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 --2- 172.21.15.107:0/4247113598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c501089b0 0x7f3c50108e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009900 tx=0x7f3c3802bc50 comp rx=0 tx=0).stop 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 -- 172.21.15.107:0/4247113598 shutdown_connections 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 --2- 172.21.15.107:0/4247113598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c50109370 0x7f3c5010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 --2- 172.21.15.107:0/4247113598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c501089b0 0x7f3c50108e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 --2- 172.21.15.107:0/4247113598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 0x7f3c50107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 -- 172.21.15.107:0/4247113598 >> 172.21.15.107:0/4247113598 conn(0x7f3c50076790 msgr2=0x7f3c50078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 -- 172.21.15.107:0/4247113598 shutdown_connections 2024-03-11T19:50:01.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.696+0000 7f3c57c00640 1 -- 172.21.15.107:0/4247113598 wait complete. 2024-03-11T19:50:01.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 Processor -- start 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 -- start start 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 0x7f3c5019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 0x7f3c5019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c501a5350 con 0x7f3c50109370 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c501a54c0 con 0x7f3c501089b0 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.697+0000 7f3c57c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c501a5680 con 0x7f3c501077c0 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c55c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 0x7f3c5019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c55200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 0x7f3c5019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c55c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41358/0 (socket says 172.21.15.107:41358) 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c55c00640 1 -- 172.21.15.107:0/652917243 learned_addr learned my addr 172.21.15.107:0/652917243 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:01.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 -- 172.21.15.107:0/652917243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 msgr2=0x7f3c5019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 0x7f3c5019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 -- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 msgr2=0x7f3c501a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 -- 172.21.15.107:0/652917243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009560 con 0x7f3c501089b0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c55c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.698+0000 7f3c47e00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 0x7f3c5019dda0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c3802c190 tx=0x7f3c38005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.699+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380043d0 con 0x7f3c501089b0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.699+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c380052d0 con 0x7f3c501089b0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.699+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c501a5900 con 0x7f3c501089b0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.699+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003c80 con 0x7f3c501089b0 2024-03-11T19:50:01.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.699+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c501a5e30 con 0x7f3c501089b0 2024-03-11T19:50:01.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.700+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5010eae0 con 0x7f3c501089b0 2024-03-11T19:50:01.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.701+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3c38005440 con 0x7f3c501089b0 2024-03-11T19:50:01.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.703+0000 7f3c4e000640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 0x7f3c340a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.703+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3c380efd00 con 0x7f3c501089b0 2024-03-11T19:50:01.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.703+0000 7f3c55200640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 0x7f3c340a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.704+0000 7f3c55200640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 0x7f3c340a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c50108810 tx=0x7f3c40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.707+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c380b8c70 con 0x7f3c501089b0 2024-03-11T19:50:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:01 smithi122 ceph-mon[36397]: pgmap v321: 129 pgs: 16 creating+peering, 97 active+clean, 16 unknown; 578 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-11T19:50:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:01 smithi122 ceph-mon[36397]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:50:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:01 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:50:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:01 smithi122 ceph-mon[36397]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:50:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:01 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:50:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:01 smithi145 ceph-mon[36730]: pgmap v321: 129 pgs: 16 creating+peering, 97 active+clean, 16 unknown; 578 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-11T19:50:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:01 smithi145 ceph-mon[36730]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:50:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:01 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:50:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:01 smithi145 ceph-mon[36730]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:50:01.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:01 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:50:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:01 smithi107 ceph-mon[31712]: pgmap v321: 129 pgs: 16 creating+peering, 97 active+clean, 16 unknown; 578 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-11T19:50:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:01 smithi107 ceph-mon[31712]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:50:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:01 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3307345594' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-11T19:50:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:01 smithi107 ceph-mon[31712]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:50:01.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:01 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-11T19:50:01.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.940+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c50113210 con 0x7f3c501089b0 2024-03-11T19:50:01.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.941+0000 7f3c4e000640 1 -- 172.21.15.107:0/652917243 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11298 (secure 0 0 0) 0x7f3c380bc330 con 0x7f3c501089b0 2024-03-11T19:50:01.946 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:01.946 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:00.914813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T19:50:01.946 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-03-11T19:50:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.943+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 msgr2=0x7f3c340a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.943+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 0x7f3c340a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c50108810 tx=0x7f3c40005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.943+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 msgr2=0x7f3c5019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.943+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 0x7f3c5019dda0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c3802c190 tx=0x7f3c38005e80 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 shutdown_connections 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f3c340a1ab0 0x7f3c340a3f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c501089b0 0x7f3c5019dda0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c50109370 0x7f3c501a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 --2- 172.21.15.107:0/652917243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3c501077c0 0x7f3c5019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 >> 172.21.15.107:0/652917243 conn(0x7f3c50076790 msgr2=0x7f3c5010bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 shutdown_connections 2024-03-11T19:50:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:01.944+0000 7f3c57c00640 1 -- 172.21.15.107:0/652917243 wait complete. 2024-03-11T19:50:01.967 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 -- 172.21.15.107:0/460118965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548109370 msgr2=0x7f054810fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 --2- 172.21.15.107:0/460118965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548109370 0x7f054810fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0540009ca0 tx=0x7f054002c100 comp rx=0 tx=0).stop 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 -- 172.21.15.107:0/460118965 shutdown_connections 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 --2- 172.21.15.107:0/460118965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481089b0 0x7f0548108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 --2- 172.21.15.107:0/460118965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548109370 0x7f054810fc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 --2- 172.21.15.107:0/460118965 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05481077c0 0x7f0548107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 -- 172.21.15.107:0/460118965 >> 172.21.15.107:0/460118965 conn(0x7f0548076790 msgr2=0x7f0548078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:02.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 -- 172.21.15.107:0/460118965 shutdown_connections 2024-03-11T19:50:02.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.170+0000 7f054fa00640 1 -- 172.21.15.107:0/460118965 wait complete. 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.171+0000 7f054fa00640 1 Processor -- start 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.171+0000 7f054fa00640 1 -- start start 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481077c0 0x7f054819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 0x7f05481a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481a52d0 con 0x7f05481089b0 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481a5440 con 0x7f05481077c0 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054fa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481a5600 con 0x7f0548109370 2024-03-11T19:50:02.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41382/0 (socket says 172.21.15.107:41382) 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 -- 172.21.15.107:0/2091651290 learned_addr learned my addr 172.21.15.107:0/2091651290 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054d000640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481077c0 0x7f054819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054da00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 0x7f05481a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 -- 172.21.15.107:0/2091651290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 msgr2=0x7f05481a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 0x7f05481a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 -- 172.21.15.107:0/2091651290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481077c0 msgr2=0x7f054819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481077c0 0x7f054819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f0547e00640 1 -- 172.21.15.107:0/2091651290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009900 con 0x7f05481089b0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.172+0000 7f054da00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 0x7f05481a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f0547e00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f053400d950 tx=0x7f053400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534009a30 con 0x7f05481089b0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0534009b90 con 0x7f05481089b0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534004a00 con 0x7f05481089b0 2024-03-11T19:50:02.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481a58e0 con 0x7f05481089b0 2024-03-11T19:50:02.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.173+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481a5da0 con 0x7f05481089b0 2024-03-11T19:50:02.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.174+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054810eae0 con 0x7f05481089b0 2024-03-11T19:50:02.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.176+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f0534009d00 con 0x7f05481089b0 2024-03-11T19:50:02.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.177+0000 7f0545600640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 0x7f05300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.177+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f05340cd0d0 con 0x7f05481089b0 2024-03-11T19:50:02.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.178+0000 7f054d000640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 0x7f05300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.178+0000 7f054d000640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 0x7f05300a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0548108810 tx=0x7f053c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:02.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.182+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0534096040 con 0x7f05481089b0 2024-03-11T19:50:02.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.380+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05481131a0 con 0x7f05481089b0 2024-03-11T19:50:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.385+0000 7f0545600640 1 -- 172.21.15.107:0/2091651290 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f0534099700 con 0x7f05481089b0 2024-03-11T19:50:02.390 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:02.391 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 msgr2=0x7f05300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 0x7f05300a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0548108810 tx=0x7f053c00b390 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 msgr2=0x7f054819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f053400d950 tx=0x7f053400de10 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 shutdown_connections 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05300a1a30 0x7f05300a3ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0548109370 0x7f05481a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05481089b0 0x7f054819dd20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 --2- 172.21.15.107:0/2091651290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05481077c0 0x7f054819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 >> 172.21.15.107:0/2091651290 conn(0x7f0548076790 msgr2=0x7f054810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 shutdown_connections 2024-03-11T19:50:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.388+0000 7f054fa00640 1 -- 172.21.15.107:0/2091651290 wait complete. 2024-03-11T19:50:02.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 -- 172.21.15.107:0/4147471602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301089b0 msgr2=0x7f1530108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 --2- 172.21.15.107:0/4147471602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301089b0 0x7f1530108e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1518009900 tx=0x7f151802bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 -- 172.21.15.107:0/4147471602 shutdown_connections 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 --2- 172.21.15.107:0/4147471602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1530109370 0x7f153010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 --2- 172.21.15.107:0/4147471602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301089b0 0x7f1530108e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 --2- 172.21.15.107:0/4147471602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f15301077c0 0x7f1530107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 -- 172.21.15.107:0/4147471602 >> 172.21.15.107:0/4147471602 conn(0x7f1530076790 msgr2=0x7f1530078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 -- 172.21.15.107:0/4147471602 shutdown_connections 2024-03-11T19:50:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.625+0000 7f1537c00640 1 -- 172.21.15.107:0/4147471602 wait complete. 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.626+0000 7f1537c00640 1 Processor -- start 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.626+0000 7f1537c00640 1 -- start start 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 0x7f153019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1530109370 0x7f15301a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301a5350 con 0x7f15301077c0 2024-03-11T19:50:02.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301a54c0 con 0x7f15301089b0 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1537c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15301a5680 con 0x7f1530109370 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41404/0 (socket says 172.21.15.107:41404) 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 -- 172.21.15.107:0/3988187766 learned_addr learned my addr 172.21.15.107:0/3988187766 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1530109370 0x7f15301a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1527e00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 0x7f153019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 -- 172.21.15.107:0/3988187766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1530109370 msgr2=0x7f15301a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1530109370 0x7f15301a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.627+0000 7f1535200640 1 -- 172.21.15.107:0/3988187766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 msgr2=0x7f153019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1535200640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 0x7f153019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1535200640 1 -- 172.21.15.107:0/3988187766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009560 con 0x7f15301077c0 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1527e00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 0x7f153019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:02.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1535200640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f152000e9c0 tx=0x7f152000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200096e0 con 0x7f15301077c0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f152000cd80 con 0x7f15301077c0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15301a5960 con 0x7f15301077c0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520010430 con 0x7f15301077c0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.628+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15301a5e40 con 0x7f15301077c0 2024-03-11T19:50:02.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.629+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153010eb60 con 0x7f15301077c0 2024-03-11T19:50:02.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.631+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1520003680 con 0x7f15301077c0 2024-03-11T19:50:02.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.633+0000 7f152e000640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 0x7f15140a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.633+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1520014070 con 0x7f15301077c0 2024-03-11T19:50:02.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.633+0000 7f1527e00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 0x7f15140a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.634+0000 7f1527e00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 0x7f15140a3f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1518000c00 tx=0x7f151802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.638+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1520096940 con 0x7f15301077c0 2024-03-11T19:50:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:02 smithi122 ceph-mon[36397]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:50:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:50:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:02 smithi122 ceph-mon[36397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/652917243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2091651290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:02 smithi145 ceph-mon[36730]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:50:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:50:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:02 smithi145 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/652917243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2091651290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:02 smithi107 ceph-mon[31712]: osdmap e90: 12 total, 12 up, 12 in 2024-03-11T19:50:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2981150002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-11T19:50:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:02 smithi107 ceph-mon[31712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/652917243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2091651290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.839+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f153002ce40 con 0x7f15301077c0 2024-03-11T19:50:02.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.843+0000 7f152e000640 1 -- 172.21.15.107:0/3988187766 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f152009a000 con 0x7f15301077c0 2024-03-11T19:50:02.848 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:02.848 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.845+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 msgr2=0x7f15140a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.845+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 0x7f15140a3f70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1518000c00 tx=0x7f151802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.845+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 msgr2=0x7f153019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.845+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f152000e9c0 tx=0x7f152000ee80 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 shutdown_connections 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f15140a1ab0 0x7f15140a3f70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15301089b0 0x7f153019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1530109370 0x7f15301a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 --2- 172.21.15.107:0/3988187766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15301077c0 0x7f153019d860 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 >> 172.21.15.107:0/3988187766 conn(0x7f1530076790 msgr2=0x7f153010bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 shutdown_connections 2024-03-11T19:50:02.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:02.846+0000 7f1537c00640 1 -- 172.21.15.107:0/3988187766 wait complete. 2024-03-11T19:50:02.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:50:03.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 -- 172.21.15.107:0/1375726570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc073b80 msgr2=0x7fbccc073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 --2- 172.21.15.107:0/1375726570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc073b80 0x7fbccc073fe0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009900 tx=0x7fbcb802bcd0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 -- 172.21.15.107:0/1375726570 shutdown_connections 2024-03-11T19:50:03.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 --2- 172.21.15.107:0/1375726570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc10b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 --2- 172.21.15.107:0/1375726570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc073b80 0x7fbccc073fe0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 --2- 172.21.15.107:0/1375726570 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc075160 0x7fbccc0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 -- 172.21.15.107:0/1375726570 >> 172.21.15.107:0/1375726570 conn(0x7fbccc0fc510 msgr2=0x7fbccc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 -- 172.21.15.107:0/1375726570 shutdown_connections 2024-03-11T19:50:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.100+0000 7fbccbe00640 1 -- 172.21.15.107:0/1375726570 wait complete. 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.101+0000 7fbccbe00640 1 Processor -- start 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 -- start start 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc073b80 0x7fbccc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1a0d20 con 0x7fbccc075160 2024-03-11T19:50:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1a0e90 con 0x7fbccc108e50 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1a1050 con 0x7fbccc073b80 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbcca000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbcca000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41410/0 (socket says 172.21.15.107:41410) 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbcca000640 1 -- 172.21.15.107:0/3481209971 learned_addr learned my addr 172.21.15.107:0/3481209971 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccaa00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc073b80 0x7fbccc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbccb400640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbcca000640 1 -- 172.21.15.107:0/3481209971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc073b80 msgr2=0x7fbccc199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.102+0000 7fbcca000640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc073b80 0x7fbccc199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcca000640 1 -- 172.21.15.107:0/3481209971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 msgr2=0x7fbccc1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcca000640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcca000640 1 -- 172.21.15.107:0/3481209971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8009560 con 0x7fbccc075160 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbccb400640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcca000640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8000c00 tx=0x7fbcb8005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb802dd20 con 0x7fbccc075160 2024-03-11T19:50:03.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb802de80 con 0x7fbccc075160 2024-03-11T19:50:03.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8003d00 con 0x7fbccc075160 2024-03-11T19:50:03.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc1a12d0 con 0x7fbccc075160 2024-03-11T19:50:03.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.103+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc1a17e0 con 0x7fbccc075160 2024-03-11T19:50:03.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.104+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc107c10 con 0x7fbccc075160 2024-03-11T19:50:03.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.106+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbcb8004df0 con 0x7fbccc075160 2024-03-11T19:50:03.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.108+0000 7fbcd0a00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 0x7fbcb00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.109+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbcb80efec0 con 0x7fbccc075160 2024-03-11T19:50:03.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.109+0000 7fbccaa00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 0x7fbcb00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.109+0000 7fbccaa00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 0x7fbcb00a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009b60 tx=0x7fbcbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.113+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcb80b8e90 con 0x7fbccc075160 2024-03-11T19:50:03.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.353+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbccc02cdd0 con 0x7fbccc075160 2024-03-11T19:50:03.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.355+0000 7fbcd0a00640 1 -- 172.21.15.107:0/3481209971 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11298 (secure 0 0 0) 0x7fbcb8043020 con 0x7fbccc075160 2024-03-11T19:50:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:03.360 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:00.914813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24590},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T19:50:03.361 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 msgr2=0x7fbcb00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 0x7fbcb00a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009b60 tx=0x7fbcbc005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 msgr2=0x7fbccc199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8000c00 tx=0x7fbcb8005e30 comp rx=0 tx=0).stop 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 shutdown_connections 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbcb00a1a30 0x7fbcb00a3ef0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc108e50 0x7fbccc1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbccc075160 0x7fbccc199770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 --2- 172.21.15.107:0/3481209971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbccc073b80 0x7fbccc199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 >> 172.21.15.107:0/3481209971 conn(0x7fbccc0fc510 msgr2=0x7fbccc104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 shutdown_connections 2024-03-11T19:50:03.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.358+0000 7fbccbe00640 1 -- 172.21.15.107:0/3481209971 wait complete. 2024-03-11T19:50:03.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 -- 172.21.15.107:0/407062166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5681033c0 msgr2=0x7fe5681037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 --2- 172.21.15.107:0/407062166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5681033c0 0x7fe5681037e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009900 tx=0x7fe55c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 -- 172.21.15.107:0/407062166 shutdown_connections 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 --2- 172.21.15.107:0/407062166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe568104f70 0x7fe56810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 --2- 172.21.15.107:0/407062166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681045b0 0x7fe568104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 --2- 172.21.15.107:0/407062166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5681033c0 0x7fe5681037e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 -- 172.21.15.107:0/407062166 >> 172.21.15.107:0/407062166 conn(0x7fe5680fe970 msgr2=0x7fe568100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 -- 172.21.15.107:0/407062166 shutdown_connections 2024-03-11T19:50:03.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.589+0000 7fe56f400640 1 -- 172.21.15.107:0/407062166 wait complete. 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.590+0000 7fe56f400640 1 Processor -- start 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.590+0000 7fe56f400640 1 -- start start 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 0x7fe5681994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5681045b0 0x7fe5681999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 0x7fe5681a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681a0fa0 con 0x7fe568104f70 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681a1110 con 0x7fe5681045b0 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681a12d0 con 0x7fe5681033c0 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 0x7fe5681994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 0x7fe5681a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 0x7fe5681994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50516/0 (socket says 172.21.15.107:50516) 2024-03-11T19:50:03.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56ca00640 1 -- 172.21.15.107:0/1106734887 learned_addr learned my addr 172.21.15.107:0/1106734887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe567e00640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5681045b0 0x7fe5681999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 -- 172.21.15.107:0/1106734887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 msgr2=0x7fe5681994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 0x7fe5681994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 -- 172.21.15.107:0/1106734887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5681045b0 msgr2=0x7fe5681999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5681045b0 0x7fe5681999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.591+0000 7fe56d400640 1 -- 172.21.15.107:0/1106734887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009560 con 0x7fe568104f70 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe56d400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 0x7fe5681a0a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe56000da10 tx=0x7fe56000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000bd70 con 0x7fe568104f70 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe560003e50 con 0x7fe568104f70 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5681a15b0 con 0x7fe568104f70 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5600106c0 con 0x7fe568104f70 2024-03-11T19:50:03.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.592+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681a1a70 con 0x7fe568104f70 2024-03-11T19:50:03.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.593+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56810a780 con 0x7fe568104f70 2024-03-11T19:50:03.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.595+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe560004000 con 0x7fe568104f70 2024-03-11T19:50:03.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.596+0000 7fe565600640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 0x7fe5500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.596+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe5600cdd10 con 0x7fe568104f70 2024-03-11T19:50:03.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.597+0000 7fe56ca00640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 0x7fe5500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.597+0000 7fe56ca00640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 0x7fe5500a3ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe55c02c840 tx=0x7fe55c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.599+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe560096c80 con 0x7fe568104f70 2024-03-11T19:50:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:03 smithi122 ceph-mon[36397]: pgmap v324: 129 pgs: 20 creating+peering, 100 active+clean, 9 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-11T19:50:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:03 smithi122 ceph-mon[36397]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:50:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:03 smithi122 ceph-mon[36397]: Cluster is now healthy 2024-03-11T19:50:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:03 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3988187766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:03 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3481209971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.781+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe56810eda0 con 0x7fe568104f70 2024-03-11T19:50:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:03 smithi145 ceph-mon[36730]: pgmap v324: 129 pgs: 20 creating+peering, 100 active+clean, 9 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-11T19:50:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:03 smithi145 ceph-mon[36730]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:50:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:03 smithi145 ceph-mon[36730]: Cluster is now healthy 2024-03-11T19:50:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:03 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3988187766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:03 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3481209971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.786+0000 7fe565600640 1 -- 172.21.15.107:0/1106734887 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7fe56009a340 con 0x7fe568104f70 2024-03-11T19:50:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:03.791 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:03.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 msgr2=0x7fe5500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 0x7fe5500a3ef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe55c02c840 tx=0x7fe55c005d90 comp rx=0 tx=0).stop 2024-03-11T19:50:03.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 msgr2=0x7fe5681a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 0x7fe5681a0a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe56000da10 tx=0x7fe56000ded0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 shutdown_connections 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe5500a1a30 0x7fe5500a3ef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5681045b0 0x7fe5681999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568104f70 0x7fe5681a0a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 --2- 172.21.15.107:0/1106734887 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5681033c0 0x7fe5681994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 >> 172.21.15.107:0/1106734887 conn(0x7fe5680fe970 msgr2=0x7fe568107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 shutdown_connections 2024-03-11T19:50:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:03.788+0000 7fe56f400640 1 -- 172.21.15.107:0/1106734887 wait complete. 2024-03-11T19:50:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:03 smithi107 ceph-mon[31712]: pgmap v324: 129 pgs: 20 creating+peering, 100 active+clean, 9 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-11T19:50:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:03 smithi107 ceph-mon[31712]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:50:03.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:03 smithi107 ceph-mon[31712]: Cluster is now healthy 2024-03-11T19:50:03.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3988187766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3481209971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:03.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-11T19:49:55.178113+0000', 'modified': '2024-03-11T19:50:00.914813+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24590}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24590': {'gid': 24590, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6839/1167549353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1167549353}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1167549353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-11T19:50:03.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-11T19:50:03.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:50:04.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 -- 172.21.15.107:0/2194972809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324104470 msgr2=0x7f93241048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 --2- 172.21.15.107:0/2194972809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324104470 0x7f93241048f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9310009900 tx=0x7f931002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 -- 172.21.15.107:0/2194972809 shutdown_connections 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 --2- 172.21.15.107:0/2194972809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 0x7f932410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 --2- 172.21.15.107:0/2194972809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324104470 0x7f93241048f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 --2- 172.21.15.107:0/2194972809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324103280 0x7f93241036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 -- 172.21.15.107:0/2194972809 >> 172.21.15.107:0/2194972809 conn(0x7f93240fe7f0 msgr2=0x7f9324100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 -- 172.21.15.107:0/2194972809 shutdown_connections 2024-03-11T19:50:04.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.020+0000 7f9323200640 1 -- 172.21.15.107:0/2194972809 wait complete. 2024-03-11T19:50:04.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.021+0000 7f9323200640 1 Processor -- start 2024-03-11T19:50:04.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.021+0000 7f9323200640 1 -- start start 2024-03-11T19:50:04.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324104470 0x7f93241998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 0x7f93241a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93241a0e60 con 0x7f9324103280 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93241a0fd0 con 0x7f9324104e30 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9323200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93241a1190 con 0x7f9324104470 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41446/0 (socket says 172.21.15.107:41446) 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 -- 172.21.15.107:0/2094457163 learned_addr learned my addr 172.21.15.107:0/2094457163 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321400640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324104470 0x7f93241998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9322800640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 0x7f93241a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 -- 172.21.15.107:0/2094457163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324104470 msgr2=0x7f93241998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324104470 0x7f93241998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 -- 172.21.15.107:0/2094457163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 msgr2=0x7f93241a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 0x7f93241a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.022+0000 7f9321e00640 1 -- 172.21.15.107:0/2094457163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310009560 con 0x7f9324103280 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f9321e00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f931c00eb70 tx=0x7f931c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f931aa00640 1 -- 172.21.15.107:0/2094457163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c003b70 con 0x7f9324103280 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f931aa00640 1 -- 172.21.15.107:0/2094457163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f931c00ee70 con 0x7f9324103280 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f931aa00640 1 -- 172.21.15.107:0/2094457163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0107f0 con 0x7f9324103280 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93241a1470 con 0x7f9324103280 2024-03-11T19:50:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.023+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93241a1950 con 0x7f9324103280 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.026+0000 7f931aa00640 1 -- 172.21.15.107:0/2094457163 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f931c004020 con 0x7f9324103280 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.026+0000 7f932ba00740 1 Processor -- start 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.026+0000 7f932ba00740 1 -- start start 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feb42c0 0x558b3feb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fe1db30 con 0x558b3feae760 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fe1b060 con 0x558b3feaf150 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f932ba00740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fe16840 con 0x558b3feb42c0 2024-03-11T19:50:04.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f9322800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f9322800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41452/0 (socket says 172.21.15.107:41452) 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f9322800640 1 -- 172.21.15.107:0/2192178486 learned_addr learned my addr 172.21.15.107:0/2192178486 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f9321e00640 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.027+0000 7f9321400640 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feb42c0 0x558b3feb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 -- 172.21.15.107:0/2192178486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feb42c0 msgr2=0x558b3feb6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feb42c0 0x558b3feb6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 -- 172.21.15.107:0/2192178486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 msgr2=0x558b3feb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 -- 172.21.15.107:0/2192178486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b3fdde4c0 con 0x558b3feae760 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f931aa00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 0x7f93080a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9322800640 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f931400dc30 tx=0x7f931400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7936b2b78745541f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f931aa00640 1 -- 172.21.15.107:0/2094457163 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f931c014070 con 0x7f9324103280 2024-03-11T19:50:04.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f9321e00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 0x7f93080a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f930f000640 1 -- 172.21.15.107:0/2192178486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314004030 con 0x558b3feae760 2024-03-11T19:50:04.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f930f000640 1 -- 172.21.15.107:0/2192178486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9314004190 con 0x558b3feae760 2024-03-11T19:50:04.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.028+0000 7f930f000640 1 -- 172.21.15.107:0/2192178486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140116a0 con 0x558b3feae760 2024-03-11T19:50:04.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 -- 172.21.15.107:0/2192178486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 msgr2=0x558b3feaeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f9321e00640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 0x7f93080a3f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cb500 tx=0x7f931c0d14e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f931400dc30 tx=0x7f931400f730 comp rx=0 tx=0).stop 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 -- 172.21.15.107:0/2192178486 shutdown_connections 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feb42c0 0x558b3feb6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 --2- 172.21.15.107:0/2192178486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3feaeb80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 -- 172.21.15.107:0/2192178486 >> 172.21.15.107:0/2192178486 conn(0x558b3fea9cd0 msgr2=0x558b3feaa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 -- 172.21.15.107:0/2192178486 shutdown_connections 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.029+0000 7f932ba00740 1 -- 172.21.15.107:0/2192178486 wait complete. 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 -- start start 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 Processor -- start 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3feb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feb42c0 0x558b3febfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fdbbea0 con 0x558b3feb42c0 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fdbbb40 con 0x558b3feaf150 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fb5ef20 con 0x558b3feae760 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f932ba00740 10 client.0 _ll_register_callbacks cb 0x558b3fdcf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f9322800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feb42c0 0x558b3febfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.030+0000 7f9322800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3feb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9322800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3feb85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50554/0 (socket says 172.21.15.107:50554) 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9322800640 1 -- 172.21.15.107:0/569631964 learned_addr learned my addr 172.21.15.107:0/569631964 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 -- 172.21.15.107:0/569631964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 msgr2=0x558b3feb85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3feb85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 -- 172.21.15.107:0/569631964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feb42c0 msgr2=0x558b3febfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feb42c0 0x558b3febfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 -- 172.21.15.107:0/569631964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931400d860 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9321400640 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb8b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f931002c170 tx=0x7f9310004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c887b36d681844a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9303400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.031+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310004250 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9310046070 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b3fe3ac40 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93100043b0 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 10 client.24698 fetch_fsmap learned FSMap version 17 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x558b3fe14ec0 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9310056020 con 0x558b3feaf150 2024-03-11T19:50:04.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f931003c040 con 0x558b3feaf150 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 10 client.24698 fetch_fsmap finished waiting for FSMap version 17 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 10 client.24698 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 20 client.24698 populate_metadata read hostname 'smithi107' 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3fec44e0 0x558b3fec8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 4 client.24698 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.032+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b3fb166c0 con 0x558b3fec44e0 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.033+0000 7f9322800640 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3fec44e0 0x558b3fec8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.033+0000 7f9322800640 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3fec44e0 0x558b3fec8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b905f757094dcf20 server_cookie=5dd5163f29fc53b4 in_seq=0 out_seq=0 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.033+0000 7f9303400640 10 client.24698 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.034+0000 7f9303400640 1 -- 172.21.15.107:0/569631964 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b3fb166c0 con 0x558b3fec44e0 2024-03-11T19:50:04.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.034+0000 7f9303400640 10 client.24698 handle_command_reply: tid=0 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.034+0000 7f932ba00740 1 client.24698 shutdown 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.034+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 msgr2=0x558b3feb8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.034+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb8b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f931002c170 tx=0x7f9310004960 comp rx=0 tx=0).stop 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 shutdown_connections 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3fec44e0 0x558b3fec8fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3feb8b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feb42c0 0x558b3febfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 --2- 172.21.15.107:0/569631964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3feb85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f9303400640 0 client.24698 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 >> 172.21.15.107:0/569631964 conn(0x558b3fea9cd0 msgr2=0x558b3fea8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 shutdown_connections 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 1 -- 172.21.15.107:0/569631964 wait complete. 2024-03-11T19:50:04.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.035+0000 7f932ba00740 20 client.24698 trim_cache size 0 max 16384 2024-03-11T19:50:04.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 Processor -- start 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 -- start start 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3febfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3fea8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3feb6e10 con 0x558b3feae760 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3fec2a50 con 0x558b3feaf150 2024-03-11T19:50:04.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f932ba00740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3feb97d0 con 0x558b3fea9cd0 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f9322800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f9321400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3febfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f9322800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:50558/0 (socket says 172.21.15.107:50558) 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f9322800640 1 -- 172.21.15.107:0/15124943 learned_addr learned my addr 172.21.15.107:0/15124943 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.052+0000 7f9322800640 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3fea8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 -- 172.21.15.107:0/15124943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 msgr2=0x558b3fea8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3fea8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 -- 172.21.15.107:0/15124943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 msgr2=0x558b3febfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3febfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 -- 172.21.15.107:0/15124943 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b3fdb18b0 con 0x558b3fea9cd0 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9321400640 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3febfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f9322800640 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f931400db50 tx=0x7f9314005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a009f9a2dda301d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f930e600640 1 -- 172.21.15.107:0/15124943 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314020040 con 0x558b3fea9cd0 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f930e600640 1 -- 172.21.15.107:0/15124943 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9314005050 con 0x558b3fea9cd0 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.053+0000 7f930e600640 1 -- 172.21.15.107:0/15124943 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314003a10 con 0x558b3fea9cd0 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- 172.21.15.107:0/15124943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 msgr2=0x558b3feb7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f931400db50 tx=0x7f9314005870 comp rx=0 tx=0).stop 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- 172.21.15.107:0/15124943 shutdown_connections 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3fea8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3feae760 0x558b3febfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 --2- 172.21.15.107:0/15124943 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3fea9cd0 0x558b3feb7fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- 172.21.15.107:0/15124943 >> 172.21.15.107:0/15124943 conn(0x558b3feaa420 msgr2=0x558b3feb78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- 172.21.15.107:0/15124943 shutdown_connections 2024-03-11T19:50:04.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- 172.21.15.107:0/15124943 wait complete. 2024-03-11T19:50:04.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 -- start start 2024-03-11T19:50:04.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.054+0000 7f932ba00740 1 Processor -- start 2024-03-11T19:50:04.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3febcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3ff1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3febfd30 con 0x558b3fea9cd0 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3febfea0 con 0x558b3feaf150 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3ff1b1a0 con 0x558b3feae760 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f932ba00740 10 client.0 _ll_register_callbacks cb 0x7f9310005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f9322800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f9322800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41476/0 (socket says 172.21.15.107:41476) 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f9322800640 1 -- 172.21.15.107:0/3973321496 learned_addr learned my addr 172.21.15.107:0/3973321496 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f9321400640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3febcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.055+0000 7f9322800640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3ff1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 -- 172.21.15.107:0/3973321496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 msgr2=0x558b3febcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3febcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 -- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 msgr2=0x558b3ff1ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3ff1ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 -- 172.21.15.107:0/3973321496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b3fe14ec0 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9322800640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f931400d540 tx=0x7f931400d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91421fc1d35fd49f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9303400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931402d8f0 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f93140051c0 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b3fec5b50 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9314005380 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f932ba00740 10 client.14589 fetch_fsmap learned FSMap version 17 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.056+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x558b3fdde4c0 con 0x558b3fea9cd0 2024-03-11T19:50:04.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b3fec5b50 con 0x558b3fea9cd0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9314042070 con 0x558b3fea9cd0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 10 client.14589 fetch_fsmap finished waiting for FSMap version 17 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 10 client.14589 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 20 client.14589 populate_metadata read hostname 'smithi107' 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3feb42c0 0x558b3ff1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 4 client.14589 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558b3fec54f0 con 0x558b3feb42c0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.057+0000 7f9321400640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3feb42c0 0x558b3ff1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f9321400640 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3feb42c0 0x558b3ff1c5c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559e3be2b863d70a server_cookie=b44cd58cef7fa7e4 in_seq=0 out_seq=0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f9303400640 10 client.14589 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f9303400640 1 -- 172.21.15.107:0/3973321496 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558b3fec54f0 con 0x558b3feb42c0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f9303400640 10 client.14589 handle_command_reply: tid=0 2024-03-11T19:50:04.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f932ba00740 1 client.14589 shutdown 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 msgr2=0x558b3febc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f931400d540 tx=0x7f931400d7c0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.058+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 shutdown_connections 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b3feb42c0 0x558b3ff1c5c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3feaf150 0x558b3ff1ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b3feae760 0x558b3febcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 --2- 172.21.15.107:0/3973321496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b3fea9cd0 0x558b3febc7d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9303400640 0 client.14589 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 >> 172.21.15.107:0/3973321496 conn(0x558b3feaa420 msgr2=0x558b3fea8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 shutdown_connections 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 1 -- 172.21.15.107:0/3973321496 wait complete. 2024-03-11T19:50:04.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f932ba00740 20 client.14589 trim_cache size 0 max 16384 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24590, 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 7.8858441250000002, 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 17, 2024-03-11T19:50:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 90, 2024-03-11T19:50:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:50:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 14.573127854000001 2024-03-11T19:50:04.070 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:50:04.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 msgr2=0x7f93080a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 0x7f93080a3f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cb500 tx=0x7f931c0d14e0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 msgr2=0x7f9324199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f931c00eb70 tx=0x7f931c00c6a0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 shutdown_connections 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93080a1ab0 0x7f93080a3f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9324104e30 0x7f93241a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9324104470 0x7f93241998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 --2- 172.21.15.107:0/2094457163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324103280 0x7f9324199370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.059+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 >> 172.21.15.107:0/2094457163 conn(0x7f93240fe7f0 msgr2=0x7f9324107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.060+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 shutdown_connections 2024-03-11T19:50:04.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:04.060+0000 7f9323200640 1 -- 172.21.15.107:0/2094457163 wait complete. 2024-03-11T19:50:04.082 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T19:50:04.082 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:04.082 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-11T19:50:04.522 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:04.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:04 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1106734887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:04.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:04 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1106734887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:04 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1106734887' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:05 smithi107 ceph-mon[31712]: pgmap v325: 129 pgs: 20 creating+peering, 109 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-11T19:50:05.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:05 smithi122 ceph-mon[36397]: pgmap v325: 129 pgs: 20 creating+peering, 109 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-11T19:50:05.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:05 smithi145 ceph-mon[36730]: pgmap v325: 129 pgs: 20 creating+peering, 109 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-11T19:50:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 -- 172.21.15.107:0/1434199406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 msgr2=0x7fc948102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 --2- 172.21.15.107:0/1434199406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948102290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009990 tx=0x7fc93c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 -- 172.21.15.107:0/1434199406 shutdown_connections 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 --2- 172.21.15.107:0/1434199406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948106f90 0x7fc94810eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 --2- 172.21.15.107:0/1434199406 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948102290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 --2- 172.21.15.107:0/1434199406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948102860 0x7fc9481068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 -- 172.21.15.107:0/1434199406 >> 172.21.15.107:0/1434199406 conn(0x7fc9480fd720 msgr2=0x7fc9480ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 -- 172.21.15.107:0/1434199406 shutdown_connections 2024-03-11T19:50:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.059+0000 7fc950000700 1 -- 172.21.15.107:0/1434199406 wait complete. 2024-03-11T19:50:07.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.060+0000 7fc950000700 1 Processor -- start 2024-03-11T19:50:07.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 -- start start 2024-03-11T19:50:07.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:07.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948102860 0x7fc9480744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:07.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:07.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481a65a0 con 0x7fc948102860 2024-03-11T19:50:07.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481a6710 con 0x7fc948106f90 2024-03-11T19:50:07.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.061+0000 7fc950000700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9481a68d0 con 0x7fc948101eb0 2024-03-11T19:50:07.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:34034/0 (socket says 172.21.15.107:34034) 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 -- 172.21.15.107:0/507215562 learned_addr learned my addr 172.21.15.107:0/507215562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94cc00700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948102860 0x7fc9480744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94d600700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 -- 172.21.15.107:0/507215562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 msgr2=0x7fc948073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 -- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948102860 msgr2=0x7fc9480744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948102860 0x7fc9480744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 -- 172.21.15.107:0/507215562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940009c40 con 0x7fc948106f90 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc94e000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c370 tx=0x7fc94000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940017070 con 0x7fc948106f90 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.062+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc94000f040 con 0x7fc948106f90 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.063+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c0095f0 con 0x7fc948106f90 2024-03-11T19:50:07.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.063+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000e050 con 0x7fc948106f90 2024-03-11T19:50:07.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.063+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9481a6ec0 con 0x7fc948106f90 2024-03-11T19:50:07.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.064+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94810a1c0 con 0x7fc948106f90 2024-03-11T19:50:07.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.066+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc94000cb60 con 0x7fc948106f90 2024-03-11T19:50:07.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.067+0000 7fc946000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 0x7fc9300a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:07.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.067+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc9400cd5d0 con 0x7fc948106f90 2024-03-11T19:50:07.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.067+0000 7fc94d600700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 0x7fc9300a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:07.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.068+0000 7fc94d600700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 0x7fc9300a3f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098e0 tx=0x7fc93c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:07.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.069+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc940096540 con 0x7fc948106f90 2024-03-11T19:50:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.336+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc948111cf0 con 0x7fc948106f90 2024-03-11T19:50:07.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.337+0000 7fc946000700 1 -- 172.21.15.107:0/507215562 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc940099c00 con 0x7fc948106f90 2024-03-11T19:50:07.342 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 msgr2=0x7fc9300a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 0x7fc9300a3f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098e0 tx=0x7fc93c005960 comp rx=0 tx=0).stop 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 msgr2=0x7fc9480749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c370 tx=0x7fc94000c680 comp rx=0 tx=0).stop 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 shutdown_connections 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc9300a1ac0 0x7fc9300a3f80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc948106f90 0x7fc9480749f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc948101eb0 0x7fc948073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 --2- 172.21.15.107:0/507215562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc948102860 0x7fc9480744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 >> 172.21.15.107:0/507215562 conn(0x7fc9480fd720 msgr2=0x7fc9480ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 shutdown_connections 2024-03-11T19:50:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:07.340+0000 7fc950000700 1 -- 172.21.15.107:0/507215562 wait complete. 2024-03-11T19:50:07.759 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:07 smithi107 ceph-mon[31712]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-11T19:50:07.759 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:07 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/507215562' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:50:07.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:07 smithi122 ceph-mon[36397]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-11T19:50:07.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:07 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/507215562' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:50:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:07 smithi145 ceph-mon[36730]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-11T19:50:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:07 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/507215562' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:50:08.424 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-11T19:50:08.424 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T19:50:08.424 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-03-11T19:50:08.424 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-11T19:50:08.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:08.425 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-11T19:50:08.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T19:50:08.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T19:50:08.425 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T19:50:08.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:08.425 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-03-11T19:50:08.481 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42059sec preferred_lft 42059sec 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-11T19:50:08.482 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:50:08.483 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T19:50:08.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T19:50:08.484 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:50:08.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:08.915 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T19:50:08.992 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-03-11T19:50:08.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:08.996 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-03-11T19:50:09.063 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-03-11T19:50:09.063 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T19:50:09.063 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T19:50:09.063 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T19:50:09.063 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T19:50:09.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:09.065 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:50:09.065 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:50:09.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T19:50:09.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T19:50:09.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T19:50:09.066 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:50:09.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:09 smithi107 ceph-mon[31712]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-11T19:50:09.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:09.760 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-11T19:50:09.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:09 smithi122 ceph-mon[36397]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-11T19:50:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:09 smithi145 ceph-mon[36730]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-11T19:50:09.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:09.815 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-03-11T19:50:09.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:09.867 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:50:09.867 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:50:09.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:09.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-11T19:50:09.867 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:50:10.009 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-11T19:50:10.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-11T19:50:10.010 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:50:10.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:10.347 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:50:10.347 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:50:10.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-03-11T19:50:10.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T19:50:10.347 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:50:10.509 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-11T19:50:10.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:50:10.509 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:10.556 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T19:50:10.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:50:10.556 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:10.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-11T19:50:10.750 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T19:50:10.792 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-11T19:50:10.792 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-11T19:50:10.792 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-11T19:50:10.792 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-11T19:50:10.793 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/0f294808690d654baec88411c3719bb87d7928d3734184011ad6943d77af79fc/merged 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a02559d886c2f215d18e7ed40cd9505c5c31457619f9e300ff8e089132b6223a/merged 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:10.794 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/fe534b5c42dede188c3ea2dccfaf2f80472a0a22f0afabc267341085c9d54d01/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/1dc504e6bb2b7aaa0a45989a352cc7a673484d6b2a26cdd988b05615937dfe14/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/9582b593b37ced29776006d67662e9e3e8c59d8a56bc44a3bf2fc3329169b4a9/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4137e389d5752fc8d4fe6bfeae85f6e5dd975f563ff910bb55036cbdd4e142bb/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/ee9327bf55bbc15940ea2ac87c2e279d03932c9b0332674c5d13996297798f02/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/af18183fabab99b58ea86a0ae3ebbeffb4ddbb4d4522c101af382816336f17c6/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5403090d1209cd46bc7703fd266a418a01bab6af09c3a9d3c269a69857258210/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/acf4ab6795f116fedd5a6eac35830a35ed76d75b5eb748042337257e85a55856/merged 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:10.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:10.797 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-11T19:50:10.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-11T19:50:10.886 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-11T19:50:10.890 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-11T19:50:10.962 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-11T19:50:11.002 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-11T19:50:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/0f294808690d654baec88411c3719bb87d7928d3734184011ad6943d77af79fc/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a02559d886c2f215d18e7ed40cd9505c5c31457619f9e300ff8e089132b6223a/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/fe534b5c42dede188c3ea2dccfaf2f80472a0a22f0afabc267341085c9d54d01/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/1dc504e6bb2b7aaa0a45989a352cc7a673484d6b2a26cdd988b05615937dfe14/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/9582b593b37ced29776006d67662e9e3e8c59d8a56bc44a3bf2fc3329169b4a9/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4137e389d5752fc8d4fe6bfeae85f6e5dd975f563ff910bb55036cbdd4e142bb/merged 2024-03-11T19:50:11.004 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/ee9327bf55bbc15940ea2ac87c2e279d03932c9b0332674c5d13996297798f02/merged 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/af18183fabab99b58ea86a0ae3ebbeffb4ddbb4d4522c101af382816336f17c6/merged 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5403090d1209cd46bc7703fd266a418a01bab6af09c3a9d3c269a69857258210/merged 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/acf4ab6795f116fedd5a6eac35830a35ed76d75b5eb748042337257e85a55856/merged 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:11.005 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:11.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:11.006 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-11T19:50:11.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:2024-03-11T19:50:11.082+0000 7f978cd3d480 -1 init, newargv = 0x563d51794390 newargc=15 2024-03-11T19:50:11.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[75254]: starting ceph client 2024-03-11T19:50:11.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[75254]: starting fuse 2024-03-11T19:50:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:11 smithi122 ceph-mon[36397]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:11 smithi145 ceph-mon[36730]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:11.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:11 smithi107 ceph-mon[31712]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:12.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-11T19:50:12.277 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T19:50:12.321 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-11T19:50:12.321 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:12.322 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/0f294808690d654baec88411c3719bb87d7928d3734184011ad6943d77af79fc/merged 2024-03-11T19:50:12.323 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a02559d886c2f215d18e7ed40cd9505c5c31457619f9e300ff8e089132b6223a/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/fe534b5c42dede188c3ea2dccfaf2f80472a0a22f0afabc267341085c9d54d01/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/1dc504e6bb2b7aaa0a45989a352cc7a673484d6b2a26cdd988b05615937dfe14/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/9582b593b37ced29776006d67662e9e3e8c59d8a56bc44a3bf2fc3329169b4a9/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4137e389d5752fc8d4fe6bfeae85f6e5dd975f563ff910bb55036cbdd4e142bb/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/ee9327bf55bbc15940ea2ac87c2e279d03932c9b0332674c5d13996297798f02/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/af18183fabab99b58ea86a0ae3ebbeffb4ddbb4d4522c101af382816336f17c6/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5403090d1209cd46bc7703fd266a418a01bab6af09c3a9d3c269a69857258210/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/acf4ab6795f116fedd5a6eac35830a35ed76d75b5eb748042337257e85a55856/merged 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:50:12.324 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:12.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:12.325 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-11T19:50:12.413 INFO:teuthology.orchestra.run.smithi107.stdout:118 2024-03-11T19:50:12.413 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-03-11T19:50:12.414 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.admin")) 2024-03-11T19:50:12.415 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-03-11T19:50:12.617 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.admin.75254.asok 2024-03-11T19:50:12.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-11T19:50:12.634 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75254.asok 2024-03-11T19:50:12.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:12.634 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75254.asok status 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "94a539707733325322964c6a187c6a1b41be1daf", 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "admin", 2024-03-11T19:50:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "75254", 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24716, 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 24716 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-11T19:50:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 827187009 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 827187009 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.24716 192.168.0.1:0/827187009", 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/827187009", 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 17, 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 90, 2024-03-11T19:50:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-03-11T19:50:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-03-11T19:50:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-03-11T19:50:12.870 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:50:12.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:12.882 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-03-11T19:50:12.882 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:12.937 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-03-11T19:50:12.937 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:12.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:12.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:13.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:13.033 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-11T19:50:13.122 INFO:teuthology.orchestra.run.smithi107.stdout:# file: . 2024-03-11T19:50:13.122 INFO:teuthology.orchestra.run.smithi107.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-11T19:50:13.122 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:13.123 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:50:13.123 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:50:13.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 -- 172.21.15.107:0/1877197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 msgr2=0x7f6d00108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 --2- 172.21.15.107:0/1877197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d00108e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009900 tx=0x7f6ce802bc40 comp rx=0 tx=0).stop 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 -- 172.21.15.107:0/1877197930 shutdown_connections 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 --2- 172.21.15.107:0/1877197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d00108e30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 --2- 172.21.15.107:0/1877197930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d00109370 0x7f6d0010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 --2- 172.21.15.107:0/1877197930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d001077c0 0x7f6d00107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 -- 172.21.15.107:0/1877197930 >> 172.21.15.107:0/1877197930 conn(0x7f6d00076790 msgr2=0x7f6d00078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 -- 172.21.15.107:0/1877197930 shutdown_connections 2024-03-11T19:50:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.373+0000 7f6d06e00640 1 -- 172.21.15.107:0/1877197930 wait complete. 2024-03-11T19:50:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.374+0000 7f6d06e00640 1 Processor -- start 2024-03-11T19:50:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 -- start start 2024-03-11T19:50:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d0019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 0x7f6d001a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a52f0 con 0x7f6d001077c0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a5460 con 0x7f6d001089b0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d06e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a5620 con 0x7f6d00109370 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45356/0 (socket says 172.21.15.107:45356) 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6cffe00640 1 -- 172.21.15.107:0/976425305 learned_addr learned my addr 172.21.15.107:0/976425305 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6cf7400640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d0019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.375+0000 7f6d04e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 0x7f6d001a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 -- 172.21.15.107:0/976425305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 msgr2=0x7f6d001a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 0x7f6d001a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 -- 172.21.15.107:0/976425305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 msgr2=0x7f6d0019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d0019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 -- 172.21.15.107:0/976425305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8009560 con 0x7f6d001077c0 2024-03-11T19:50:13.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6d04e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 0x7f6d001a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:13.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cffe00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000ebc0 tx=0x7f6cf000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:13.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0003b70 con 0x7f6d001077c0 2024-03-11T19:50:13.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cf000eec0 con 0x7f6d001077c0 2024-03-11T19:50:13.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d001a5900 con 0x7f6d001077c0 2024-03-11T19:50:13.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0019800 con 0x7f6d001077c0 2024-03-11T19:50:13.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.376+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d001a5de0 con 0x7f6d001077c0 2024-03-11T19:50:13.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.378+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0010eae0 con 0x7f6d001077c0 2024-03-11T19:50:13.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.380+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6cf0019960 con 0x7f6d001077c0 2024-03-11T19:50:13.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.382+0000 7f6cfd600640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 0x7f6ce40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.382+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6cf0014070 con 0x7f6d001077c0 2024-03-11T19:50:13.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.382+0000 7f6cf7400640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 0x7f6ce40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.383+0000 7f6cf7400640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 0x7f6ce40a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009230 tx=0x7f6ce8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:13.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.387+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6cf0096f00 con 0x7f6d001077c0 2024-03-11T19:50:13.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:13 smithi107 ceph-mon[31712]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:13.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.585+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d0002ce70 con 0x7f6d001077c0 2024-03-11T19:50:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.590+0000 7f6cfd600640 1 -- 172.21.15.107:0/976425305 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f6cf009a5c0 con 0x7f6d001077c0 2024-03-11T19:50:13.595 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:13.595 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:13.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.592+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 msgr2=0x7f6ce40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.592+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 0x7f6ce40a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009230 tx=0x7f6ce8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.592+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 msgr2=0x7f6d0019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.592+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000ebc0 tx=0x7f6cf000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 shutdown_connections 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6ce40a1a30 0x7f6ce40a3ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d001089b0 0x7f6d0019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d00109370 0x7f6d001a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 --2- 172.21.15.107:0/976425305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d001077c0 0x7f6d0019d800 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 >> 172.21.15.107:0/976425305 conn(0x7f6d00076790 msgr2=0x7f6d0010bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 shutdown_connections 2024-03-11T19:50:13.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.593+0000 7f6d06e00640 1 -- 172.21.15.107:0/976425305 wait complete. 2024-03-11T19:50:13.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:50:13.615 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:50:13.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:13 smithi122 ceph-mon[36397]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:13 smithi145 ceph-mon[36730]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:50:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.824+0000 7f920f200640 1 -- 172.21.15.107:0/3030842267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210103280 msgr2=0x7f92101036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.824+0000 7f920f200640 1 --2- 172.21.15.107:0/3030842267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210103280 0x7f92101036a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9200009900 tx=0x7f920002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 -- 172.21.15.107:0/3030842267 shutdown_connections 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 --2- 172.21.15.107:0/3030842267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210104e30 0x7f921010b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 --2- 172.21.15.107:0/3030842267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 0x7f92101048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 --2- 172.21.15.107:0/3030842267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210103280 0x7f92101036a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 -- 172.21.15.107:0/3030842267 >> 172.21.15.107:0/3030842267 conn(0x7f92100fe7f0 msgr2=0x7f9210100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 -- 172.21.15.107:0/3030842267 shutdown_connections 2024-03-11T19:50:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.825+0000 7f920f200640 1 -- 172.21.15.107:0/3030842267 wait complete. 2024-03-11T19:50:13.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 Processor -- start 2024-03-11T19:50:13.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 -- start start 2024-03-11T19:50:13.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210103280 0x7f9210199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 0x7f9210199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101a0e30 con 0x7f9210104e30 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101a0fa0 con 0x7f9210103280 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101a1160 con 0x7f9210104470 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45366/0 (socket says 172.21.15.107:45366) 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920e800640 1 -- 172.21.15.107:0/2411209034 learned_addr learned my addr 172.21.15.107:0/2411209034 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f9205400640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 0x7f9210199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.826+0000 7f920de00640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210103280 0x7f9210199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 -- 172.21.15.107:0/2411209034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 msgr2=0x7f9210199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 0x7f9210199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 -- 172.21.15.107:0/2411209034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210103280 msgr2=0x7f9210199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:13.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210103280 0x7f9210199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 -- 172.21.15.107:0/2411209034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009560 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920e800640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f920800e970 tx=0x7f920800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208009720 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9208005500 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92101a1440 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208010430 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.827+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92101a1900 con 0x7f9210104e30 2024-03-11T19:50:13.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.829+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921010a5f0 con 0x7f9210104e30 2024-03-11T19:50:13.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.830+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9208005020 con 0x7f9210104e30 2024-03-11T19:50:13.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.832+0000 7f9207400640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 0x7f91f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:13.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.832+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9208014070 con 0x7f9210104e30 2024-03-11T19:50:13.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.832+0000 7f920de00640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 0x7f91f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:13.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.833+0000 7f920de00640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 0x7f91f40a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f920002c240 tx=0x7f920002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:13.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:13.838+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92080969a0 con 0x7f9210104e30 2024-03-11T19:50:14.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.038+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f921002cda0 con 0x7f9210104e30 2024-03-11T19:50:14.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.039+0000 7f9207400640 1 -- 172.21.15.107:0/2411209034 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f920809a060 con 0x7f9210104e30 2024-03-11T19:50:14.044 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 msgr2=0x7f91f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 0x7f91f40a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f920002c240 tx=0x7f920002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 msgr2=0x7f92101a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f920800e970 tx=0x7f920800ed70 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 shutdown_connections 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f91f40a1a30 0x7f91f40a3ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9210104e30 0x7f92101a08f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9210104470 0x7f9210199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 --2- 172.21.15.107:0/2411209034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9210103280 0x7f9210199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 >> 172.21.15.107:0/2411209034 conn(0x7f92100fe7f0 msgr2=0x7f92101077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:14.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 shutdown_connections 2024-03-11T19:50:14.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.042+0000 7f920f200640 1 -- 172.21.15.107:0/2411209034 wait complete. 2024-03-11T19:50:14.064 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.265+0000 7f2d0ae00640 1 -- 172.21.15.107:0/678464298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 msgr2=0x7f2d04108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.265+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/678464298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d04108d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009900 tx=0x7f2cf402bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 -- 172.21.15.107:0/678464298 shutdown_connections 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/678464298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d04108d10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/678464298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 0x7f2d0410fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/678464298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d04107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 -- 172.21.15.107:0/678464298 >> 172.21.15.107:0/678464298 conn(0x7f2d04076650 msgr2=0x7f2d04078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 -- 172.21.15.107:0/678464298 shutdown_connections 2024-03-11T19:50:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.266+0000 7f2d0ae00640 1 -- 172.21.15.107:0/678464298 wait complete. 2024-03-11T19:50:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 Processor -- start 2024-03-11T19:50:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 -- start start 2024-03-11T19:50:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d0419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 0x7f2d041a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041a5190 con 0x7f2d041076a0 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041a5300 con 0x7f2d04108890 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041a54c0 con 0x7f2d04109250 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d09a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d09a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45392/0 (socket says 172.21.15.107:45392) 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d09a00640 1 -- 172.21.15.107:0/475240441 learned_addr learned my addr 172.21.15.107:0/475240441 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d09000640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d0419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.267+0000 7f2d0a400640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 0x7f2d041a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 -- 172.21.15.107:0/475240441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 msgr2=0x7f2d041a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 0x7f2d041a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 -- 172.21.15.107:0/475240441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 msgr2=0x7f2d0419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d0419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 -- 172.21.15.107:0/475240441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4009560 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d09a00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800e9c0 tx=0x7f2cf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8009700 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d041a57a0 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cf800cd80 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.268+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8010430 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.269+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d041a5cd0 con 0x7f2d041076a0 2024-03-11T19:50:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.270+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0410e960 con 0x7f2d041076a0 2024-03-11T19:50:14.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.272+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f2cf8003680 con 0x7f2d041076a0 2024-03-11T19:50:14.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.273+0000 7f2d02000640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 0x7f2cec0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.273+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2cf8014070 con 0x7f2d041076a0 2024-03-11T19:50:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.273+0000 7f2d09000640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 0x7f2cec0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.274+0000 7f2d09000640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 0x7f2cec0a3ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000c00 tx=0x7f2cf402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.277+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cf80969c0 con 0x7f2d041076a0 2024-03-11T19:50:14.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.477+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d041130d0 con 0x7f2d041076a0 2024-03-11T19:50:14.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.478+0000 7f2d02000640 1 -- 172.21.15.107:0/475240441 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cf809a080 con 0x7f2d041076a0 2024-03-11T19:50:14.483 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 msgr2=0x7f2cec0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 0x7f2cec0a3ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000c00 tx=0x7f2cf402c7e0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 msgr2=0x7f2d0419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800e9c0 tx=0x7f2cf800ee80 comp rx=0 tx=0).stop 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 shutdown_connections 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2cec0a1a30 0x7f2cec0a3ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d04108890 0x7f2d0419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d04109250 0x7f2d041a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 --2- 172.21.15.107:0/475240441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d041076a0 0x7f2d0419d6a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.481+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 >> 172.21.15.107:0/475240441 conn(0x7f2d04076650 msgr2=0x7f2d0410bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:14.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.482+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 shutdown_connections 2024-03-11T19:50:14.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.482+0000 7f2d0ae00640 1 -- 172.21.15.107:0/475240441 wait complete. 2024-03-11T19:50:14.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:50:14.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.711+0000 7fbbc9e00640 1 -- 172.21.15.107:0/2586622819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc4108990 msgr2=0x7fbbc4108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.711+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/2586622819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc4108990 0x7fbbc4108e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009860 tx=0x7fbbac02bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 -- 172.21.15.107:0/2586622819 shutdown_connections 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/2586622819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc4108990 0x7fbbc4108e10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/2586622819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4109350 0x7fbbc410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/2586622819 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc41077a0 0x7fbbc4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 -- 172.21.15.107:0/2586622819 >> 172.21.15.107:0/2586622819 conn(0x7fbbc4076740 msgr2=0x7fbbc4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 -- 172.21.15.107:0/2586622819 shutdown_connections 2024-03-11T19:50:14.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.712+0000 7fbbc9e00640 1 -- 172.21.15.107:0/2586622819 wait complete. 2024-03-11T19:50:14.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 Processor -- start 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 -- start start 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 0x7fbbc419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41a5330 con 0x7fbbc4108990 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41a54a0 con 0x7fbbc41077a0 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.713+0000 7fbbc9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41a5660 con 0x7fbbc4109350 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 0x7fbbc419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 0x7fbbc419d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:34106/0 (socket says 172.21.15.107:34106) 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45400/0 (socket says 172.21.15.107:45400) 2024-03-11T19:50:14.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37606/0 (socket says 172.21.15.107:37606) 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc2a00640 1 -- 172.21.15.107:0/4276642599 learned_addr learned my addr 172.21.15.107:0/4276642599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 -- 172.21.15.107:0/4276642599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 msgr2=0x7fbbc419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 0x7fbbc419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 -- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 msgr2=0x7fbbc419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 -- 172.21.15.107:0/4276642599 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009490 con 0x7fbbc4109350 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbba000640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.714+0000 7fbbc3400640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00ef00 tx=0x7fbbbc00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.715+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc004b20 con 0x7fbbc4109350 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.715+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbbc003e50 con 0x7fbbc4109350 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.715+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc41a5940 con 0x7fbbc4109350 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.715+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00b570 con 0x7fbbc4109350 2024-03-11T19:50:14.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.715+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc41a5dd0 con 0x7fbbc4109350 2024-03-11T19:50:14.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.716+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc410eac0 con 0x7fbbc4109350 2024-03-11T19:50:14.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.718+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbbbc00a040 con 0x7fbbc4109350 2024-03-11T19:50:14.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.719+0000 7fbbc8a00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 0x7fbba80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:14.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.719+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbbbc0cd520 con 0x7fbbc4109350 2024-03-11T19:50:14.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.719+0000 7fbbc2a00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 0x7fbba80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:14.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.720+0000 7fbbc2a00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 0x7fbba80a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40058c0 tx=0x7fbbb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:14.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.724+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbbc096490 con 0x7fbbc4109350 2024-03-11T19:50:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:14 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/976425305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:14.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:14 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2411209034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:14.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:14 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/976425305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:14.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:14 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2411209034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:14.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:14 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/976425305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:14.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:14 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2411209034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:14.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.918+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbc402ce40 con 0x7fbbc4109350 2024-03-11T19:50:14.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.919+0000 7fbbc8a00640 1 -- 172.21.15.107:0/4276642599 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbbbc01a0a0 con 0x7fbbc4109350 2024-03-11T19:50:14.924 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:50:14.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.921+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 msgr2=0x7fbba80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.921+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 0x7fbba80a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40058c0 tx=0x7fbbb4005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:14.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.921+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 msgr2=0x7fbbc41a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:14.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.921+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00ef00 tx=0x7fbbbc00c740 comp rx=0 tx=0).stop 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 shutdown_connections 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbba80a1a30 0x7fbba80a3ef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbbc4109350 0x7fbbc41a4df0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbc4108990 0x7fbbc419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 --2- 172.21.15.107:0/4276642599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc41077a0 0x7fbbc419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 >> 172.21.15.107:0/4276642599 conn(0x7fbbc4076740 msgr2=0x7fbbc410bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 shutdown_connections 2024-03-11T19:50:14.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:14.922+0000 7fbbc9e00640 1 -- 172.21.15.107:0/4276642599 wait complete. 2024-03-11T19:50:14.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:50:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 -- 172.21.15.107:0/2178017655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 msgr2=0x7fc1481037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 --2- 172.21.15.107:0/2178017655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc1481037e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc140009860 tx=0x7fc14002bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 -- 172.21.15.107:0/2178017655 shutdown_connections 2024-03-11T19:50:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 --2- 172.21.15.107:0/2178017655 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148104f70 0x7fc14810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 --2- 172.21.15.107:0/2178017655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1481045b0 0x7fc148104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 --2- 172.21.15.107:0/2178017655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc1481037e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 -- 172.21.15.107:0/2178017655 >> 172.21.15.107:0/2178017655 conn(0x7fc1480fe930 msgr2=0x7fc148100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 -- 172.21.15.107:0/2178017655 shutdown_connections 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.157+0000 7fc14d400640 1 -- 172.21.15.107:0/2178017655 wait complete. 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 Processor -- start 2024-03-11T19:50:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 -- start start 2024-03-11T19:50:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1481045b0 0x7fc1481999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc148104f70 0x7fc1481a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1481a0f50 con 0x7fc148104f70 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1481a10c0 con 0x7fc1481033c0 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.159+0000 7fc14d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1481a1280 con 0x7fc1481045b0 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc145e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1481045b0 0x7fc1481999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36048/0 (socket says 172.21.15.107:36048) 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 -- 172.21.15.107:0/3620506584 learned_addr learned my addr 172.21.15.107:0/3620506584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc147200640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc148104f70 0x7fc1481a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 -- 172.21.15.107:0/3620506584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1481045b0 msgr2=0x7fc1481999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1481045b0 0x7fc1481999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 -- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc148104f70 msgr2=0x7fc1481a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc148104f70 0x7fc1481a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 -- 172.21.15.107:0/3620506584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140009490 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.160+0000 7fc146800640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc14002c0b0 tx=0x7fc140005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.161+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14002c510 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.161+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc14002cb50 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.161+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1481a1500 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.161+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1400035f0 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.161+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1481a1870 con 0x7fc1481033c0 2024-03-11T19:50:15.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.162+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14810a760 con 0x7fc1481033c0 2024-03-11T19:50:15.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.164+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc14003b040 con 0x7fc1481033c0 2024-03-11T19:50:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.166+0000 7fc13f400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 0x7fc12c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.166+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc1400ef200 con 0x7fc1481033c0 2024-03-11T19:50:15.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.166+0000 7fc145e00640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 0x7fc12c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.167+0000 7fc145e00640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 0x7fc12c0a3ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc14819a970 tx=0x7fc130009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.170+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1400b8170 con 0x7fc1481033c0 2024-03-11T19:50:15.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.371+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc14802cdd0 con 0x7fc1481033c0 2024-03-11T19:50:15.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.372+0000 7fc13f400640 1 -- 172.21.15.107:0/3620506584 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc1400bb830 con 0x7fc1481033c0 2024-03-11T19:50:15.377 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 msgr2=0x7fc12c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 0x7fc12c0a3ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc14819a970 tx=0x7fc130009260 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 msgr2=0x7fc148199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc14002c0b0 tx=0x7fc140005d50 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 shutdown_connections 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc12c0a1a30 0x7fc12c0a3ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc148104f70 0x7fc1481a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1481045b0 0x7fc1481999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 --2- 172.21.15.107:0/3620506584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1481033c0 0x7fc148199460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 >> 172.21.15.107:0/3620506584 conn(0x7fc1480fe930 msgr2=0x7fc148107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:15.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 shutdown_connections 2024-03-11T19:50:15.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.375+0000 7fc14d400640 1 -- 172.21.15.107:0/3620506584 wait complete. 2024-03-11T19:50:15.399 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:50:15.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 -- 172.21.15.107:0/2946696078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 msgr2=0x7faf680771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 --2- 172.21.15.107:0/2946696078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf680771e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf50009900 tx=0x7faf5002bc50 comp rx=0 tx=0).stop 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 -- 172.21.15.107:0/2946696078 shutdown_connections 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 --2- 172.21.15.107:0/2946696078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf680771e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 --2- 172.21.15.107:0/2946696078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 0x7faf68077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 --2- 172.21.15.107:0/2946696078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf68075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.598+0000 7faf6ce00640 1 -- 172.21.15.107:0/2946696078 >> 172.21.15.107:0/2946696078 conn(0x7faf680fe930 msgr2=0x7faf68100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.599+0000 7faf6ce00640 1 -- 172.21.15.107:0/2946696078 shutdown_connections 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.599+0000 7faf6ce00640 1 -- 172.21.15.107:0/2946696078 wait complete. 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.599+0000 7faf6ce00640 1 Processor -- start 2024-03-11T19:50:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 -- start start 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 0x7faf681a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a5330 con 0x7faf68075b40 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a54a0 con 0x7faf68076d80 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf6ce00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a5660 con 0x7faf68077720 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf65e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf5d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf5d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36054/0 (socket says 172.21.15.107:36054) 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf65e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46626/0 (socket says 172.21.15.107:46626) 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf5d400640 1 -- 172.21.15.107:0/346206150 learned_addr learned my addr 172.21.15.107:0/346206150 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.600+0000 7faf66800640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 0x7faf681a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 -- 172.21.15.107:0/346206150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 msgr2=0x7faf681a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 0x7faf681a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 -- 172.21.15.107:0/346206150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 msgr2=0x7faf6819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 -- 172.21.15.107:0/346206150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50009560 con 0x7faf68075b40 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf5d400640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:15.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf65e00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9f0 tx=0x7faf5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800ccc0 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf681a5940 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf5800ce20 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58018920 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.601+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681a5ea0 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.603+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6810fd30 con 0x7faf68075b40 2024-03-11T19:50:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.605+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7faf58018a80 con 0x7faf68075b40 2024-03-11T19:50:15.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.606+0000 7faf5f400640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 0x7faf4c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:15.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.606+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faf58014070 con 0x7faf68075b40 2024-03-11T19:50:15.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.606+0000 7faf5d400640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 0x7faf4c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:15.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.607+0000 7faf5d400640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 0x7faf4c0a3f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf5002cdf0 tx=0x7faf5002c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:15.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.608+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf580968e0 con 0x7faf68075b40 2024-03-11T19:50:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:15 smithi122 ceph-mon[36397]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/475240441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4276642599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3620506584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:15 smithi145 ceph-mon[36730]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/475240441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4276642599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3620506584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:15 smithi107 ceph-mon[31712]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/475240441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4276642599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3620506584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:15.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.819+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faf68113210 con 0x7faf68075b40 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.820+0000 7faf5f400640 1 -- 172.21.15.107:0/346206150 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+598 (secure 0 0 0) 0x7faf58099fa0 con 0x7faf68075b40 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1356730303 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/865925559 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/531077471 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:15.825 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1620686143 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:15.826 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:15.826 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:15.826 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2887016959 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:15.826 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3318828522 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:15.826 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-03-11T19:50:15.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 msgr2=0x7faf4c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 0x7faf4c0a3f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf5002cdf0 tx=0x7faf5002c800 comp rx=0 tx=0).stop 2024-03-11T19:50:15.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 msgr2=0x7faf6819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:15.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9f0 tx=0x7faf5800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 shutdown_connections 2024-03-11T19:50:15.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7faf4c0a1ab0 0x7faf4c0a3f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf68076d80 0x7faf6819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faf68077720 0x7faf681a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 --2- 172.21.15.107:0/346206150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf68075b40 0x7faf6819d900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 >> 172.21.15.107:0/346206150 conn(0x7faf680fe930 msgr2=0x7faf68100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 shutdown_connections 2024-03-11T19:50:15.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:15.823+0000 7faf6ce00640 1 -- 172.21.15.107:0/346206150 wait complete. 2024-03-11T19:50:15.844 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:50:15.845 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:50:15.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 -- 172.21.15.107:0/1084908969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63841033c0 msgr2=0x7f63841037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 --2- 172.21.15.107:0/1084908969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63841033c0 0x7f63841037e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6378009900 tx=0x7f637802bc40 comp rx=0 tx=0).stop 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 -- 172.21.15.107:0/1084908969 shutdown_connections 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 --2- 172.21.15.107:0/1084908969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6384104f70 0x7f638410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 --2- 172.21.15.107:0/1084908969 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841045b0 0x7f6384104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 --2- 172.21.15.107:0/1084908969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63841033c0 0x7f63841037e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 -- 172.21.15.107:0/1084908969 >> 172.21.15.107:0/1084908969 conn(0x7f63840fe970 msgr2=0x7f6384100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 -- 172.21.15.107:0/1084908969 shutdown_connections 2024-03-11T19:50:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.140+0000 7f638c200640 1 -- 172.21.15.107:0/1084908969 wait complete. 2024-03-11T19:50:16.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 Processor -- start 2024-03-11T19:50:16.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 -- start start 2024-03-11T19:50:16.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63841045b0 0x7f63841999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6384104f70 0x7f63841a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841a0f70 con 0x7f6384104f70 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841a10e0 con 0x7f63841045b0 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.141+0000 7f638c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841a12a0 con 0x7f63841033c0 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37532/0 (socket says 172.21.15.107:37532) 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 -- 172.21.15.107:0/3879880941 learned_addr learned my addr 172.21.15.107:0/3879880941 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6388e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63841045b0 0x7f63841999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f638a200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6384104f70 0x7f63841a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 -- 172.21.15.107:0/3879880941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63841045b0 msgr2=0x7f63841999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63841045b0 0x7f63841999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 -- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6384104f70 msgr2=0x7f63841a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6384104f70 0x7f63841a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 -- 172.21.15.107:0/3879880941 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378009560 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.142+0000 7f6389800640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6378005ab0 tx=0x7f6378005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.143+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780043d0 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.143+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63780052d0 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.143+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378003d40 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.143+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63841a1520 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.143+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63841a1a30 con 0x7f63841033c0 2024-03-11T19:50:16.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.144+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638410a6e0 con 0x7f63841033c0 2024-03-11T19:50:16.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.146+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6378005440 con 0x7f63841033c0 2024-03-11T19:50:16.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.147+0000 7f6382000640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 0x7f636c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.147+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f63780efdb0 con 0x7f63841033c0 2024-03-11T19:50:16.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.147+0000 7f6388e00640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 0x7f636c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.148+0000 7f6388e00640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 0x7f636c0a3ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f638419a990 tx=0x7f6374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:16.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.149+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63780b8d20 con 0x7f63841033c0 2024-03-11T19:50:16.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.347+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f638402cdd0 con 0x7f63841033c0 2024-03-11T19:50:16.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.350+0000 7f6382000640 1 -- 172.21.15.107:0/3879880941 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f63780bc3e0 con 0x7f63841033c0 2024-03-11T19:50:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:16.355 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:16.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 msgr2=0x7f636c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 0x7f636c0a3ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f638419a990 tx=0x7f6374005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:16.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 msgr2=0x7f6384199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6378005ab0 tx=0x7f6378005e50 comp rx=0 tx=0).stop 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 shutdown_connections 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f636c0a1a30 0x7f636c0a3ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63841045b0 0x7f63841999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6384104f70 0x7f63841a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.352+0000 7f638c200640 1 --2- 172.21.15.107:0/3879880941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63841033c0 0x7f6384199480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.353+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 >> 172.21.15.107:0/3879880941 conn(0x7f63840fe970 msgr2=0x7f63841078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.353+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 shutdown_connections 2024-03-11T19:50:16.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.353+0000 7f638c200640 1 -- 172.21.15.107:0/3879880941 wait complete. 2024-03-11T19:50:16.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:50:16.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 -- 172.21.15.107:0/4037189844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102250 msgr2=0x7fae641026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 --2- 172.21.15.107:0/4037189844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102250 0x7fae641026d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae50009900 tx=0x7fae5002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 -- 172.21.15.107:0/4037189844 shutdown_connections 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 --2- 172.21.15.107:0/4037189844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102250 0x7fae641026d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 --2- 172.21.15.107:0/4037189844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64102c10 0x7fae641094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 --2- 172.21.15.107:0/4037189844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64101060 0x7fae64101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 -- 172.21.15.107:0/4037189844 >> 172.21.15.107:0/4037189844 conn(0x7fae640fc5f0 msgr2=0x7fae640fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:16.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 -- 172.21.15.107:0/4037189844 shutdown_connections 2024-03-11T19:50:16.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.592+0000 7fae69000640 1 -- 172.21.15.107:0/4037189844 wait complete. 2024-03-11T19:50:16.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.593+0000 7fae69000640 1 Processor -- start 2024-03-11T19:50:16.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.593+0000 7fae69000640 1 -- start start 2024-03-11T19:50:16.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64102250 0x7fae64073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 0x7fae64073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64074210 con 0x7fae64101060 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64074380 con 0x7fae64102c10 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae69000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641a1150 con 0x7fae64102250 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46652/0 (socket says 172.21.15.107:46652) 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 -- 172.21.15.107:0/3899217519 learned_addr learned my addr 172.21.15.107:0/3899217519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae62a00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64102250 0x7fae64073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63e00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 0x7fae64073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 -- 172.21.15.107:0/3899217519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64102250 msgr2=0x7fae64073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64102250 0x7fae64073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 -- 172.21.15.107:0/3899217519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 msgr2=0x7fae64073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 0x7fae64073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63400640 1 -- 172.21.15.107:0/3899217519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009560 con 0x7fae64101060 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.594+0000 7fae63e00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 0x7fae64073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:16.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae63400640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540096e0 con 0x7fae64101060 2024-03-11T19:50:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae5400cd80 con 0x7fae64101060 2024-03-11T19:50:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54010430 con 0x7fae64101060 2024-03-11T19:50:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae641a1400 con 0x7fae64101060 2024-03-11T19:50:16.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.595+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641a1890 con 0x7fae64101060 2024-03-11T19:50:16.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.596+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae641083d0 con 0x7fae64101060 2024-03-11T19:50:16.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.598+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fae54003680 con 0x7fae64101060 2024-03-11T19:50:16.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.599+0000 7fae5be00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 0x7fae480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:16.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.600+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fae54014070 con 0x7fae64101060 2024-03-11T19:50:16.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.600+0000 7fae62a00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 0x7fae480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.600+0000 7fae62a00640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 0x7fae480a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae50009230 tx=0x7fae50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:16.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.603+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae54096a10 con 0x7fae64101060 2024-03-11T19:50:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:16 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/346206150' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:16.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:16 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3879880941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:16 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/346206150' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:16.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:16 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3879880941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:16.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:16 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/346206150' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:16.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:16 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3879880941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:16.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.800+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae64107e40 con 0x7fae64101060 2024-03-11T19:50:16.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.802+0000 7fae5be00640 1 -- 172.21.15.107:0/3899217519 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae5409a0d0 con 0x7fae64101060 2024-03-11T19:50:16.806 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 msgr2=0x7fae480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 0x7fae480a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae50009230 tx=0x7fae50005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 msgr2=0x7fae64074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 shutdown_connections 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae480a1a30 0x7fae480a3ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae64102c10 0x7fae64073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae64102250 0x7fae64073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 --2- 172.21.15.107:0/3899217519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae64101060 0x7fae64074f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:16.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.804+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 >> 172.21.15.107:0/3899217519 conn(0x7fae640fc5f0 msgr2=0x7fae641055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:16.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.805+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 shutdown_connections 2024-03-11T19:50:16.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:16.805+0000 7fae69000640 1 -- 172.21.15.107:0/3899217519 wait complete. 2024-03-11T19:50:16.829 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 -- 172.21.15.107:0/814422863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8101190 msgr2=0x7fb3c81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 --2- 172.21.15.107:0/814422863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8101190 0x7fb3c81015b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009860 tx=0x7fb3b802bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 -- 172.21.15.107:0/814422863 shutdown_connections 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 --2- 172.21.15.107:0/814422863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 0x7fb3c81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 --2- 172.21.15.107:0/814422863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8102380 0x7fb3c8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 --2- 172.21.15.107:0/814422863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8101190 0x7fb3c81015b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 -- 172.21.15.107:0/814422863 >> 172.21.15.107:0/814422863 conn(0x7fb3c80fc720 msgr2=0x7fb3c80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 -- 172.21.15.107:0/814422863 shutdown_connections 2024-03-11T19:50:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.039+0000 7fb3cea00640 1 -- 172.21.15.107:0/814422863 wait complete. 2024-03-11T19:50:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.040+0000 7fb3cea00640 1 Processor -- start 2024-03-11T19:50:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 -- start start 2024-03-11T19:50:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8102380 0x7fb3c81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 0x7fb3c819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819ed80 con 0x7fb3c8101190 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819eef0 con 0x7fb3c8102d40 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819f0b0 con 0x7fb3c8102380 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46660/0 (socket says 172.21.15.107:46660) 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3c7e00640 1 -- 172.21.15.107:0/572271048 learned_addr learned my addr 172.21.15.107:0/572271048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3cca00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 0x7fb3c819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.041+0000 7fb3bf400640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8102380 0x7fb3c81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 -- 172.21.15.107:0/572271048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8102380 msgr2=0x7fb3c81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8102380 0x7fb3c81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 -- 172.21.15.107:0/572271048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 msgr2=0x7fb3c819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 0x7fb3c819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 -- 172.21.15.107:0/572271048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8009490 con 0x7fb3c8101190 2024-03-11T19:50:17.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c7e00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb3b802c0b0 tx=0x7fb3b80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b802c510 con 0x7fb3c8101190 2024-03-11T19:50:17.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3b802cb50 con 0x7fb3c8101190 2024-03-11T19:50:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c819f330 con 0x7fb3c8101190 2024-03-11T19:50:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.042+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8032600 con 0x7fb3c8101190 2024-03-11T19:50:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.043+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c819f6f0 con 0x7fb3c8101190 2024-03-11T19:50:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.044+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c8108500 con 0x7fb3c8101190 2024-03-11T19:50:17.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.046+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fb3b803b040 con 0x7fb3c8101190 2024-03-11T19:50:17.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.048+0000 7fb3c5600640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 0x7fb3ac0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.048+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb3b80ef1a0 con 0x7fb3c8101190 2024-03-11T19:50:17.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.048+0000 7fb3bf400640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 0x7fb3ac0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.049+0000 7fb3bf400640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 0x7fb3ac0a3f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3c81987a0 tx=0x7fb3b0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.052+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3b80b8220 con 0x7fb3c8101190 2024-03-11T19:50:17.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.247+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3c802cdd0 con 0x7fb3c8101190 2024-03-11T19:50:17.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.248+0000 7fb3c5600640 1 -- 172.21.15.107:0/572271048 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3b80bb8e0 con 0x7fb3c8101190 2024-03-11T19:50:17.254 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 msgr2=0x7fb3ac0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 0x7fb3ac0a3f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3c81987a0 tx=0x7fb3b0010040 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 msgr2=0x7fb3c8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb3b802c0b0 tx=0x7fb3b80045a0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 shutdown_connections 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb3ac0a1ab0 0x7fb3ac0a3f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3c8102d40 0x7fb3c819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3c8102380 0x7fb3c81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 --2- 172.21.15.107:0/572271048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3c8101190 0x7fb3c8197290 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 >> 172.21.15.107:0/572271048 conn(0x7fb3c80fc720 msgr2=0x7fb3c81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:17.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 shutdown_connections 2024-03-11T19:50:17.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.251+0000 7fb3cea00640 1 -- 172.21.15.107:0/572271048 wait complete. 2024-03-11T19:50:17.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 -- 172.21.15.107:0/3714196964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4101060 msgr2=0x7f8ae4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/3714196964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4101060 0x7f8ae4101480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009900 tx=0x7f8ad402bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 -- 172.21.15.107:0/3714196964 shutdown_connections 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/3714196964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4102c10 0x7f8ae41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/3714196964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 0x7f8ae41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/3714196964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4101060 0x7f8ae4101480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 -- 172.21.15.107:0/3714196964 >> 172.21.15.107:0/3714196964 conn(0x7f8ae40fc5f0 msgr2=0x7f8ae40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 -- 172.21.15.107:0/3714196964 shutdown_connections 2024-03-11T19:50:17.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.486+0000 7f8ae8a00640 1 -- 172.21.15.107:0/3714196964 wait complete. 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.487+0000 7f8ae8a00640 1 Processor -- start 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 -- start start 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 0x7f8ae4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4102c10 0x7f8ae419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419ec40 con 0x7f8ae4102c10 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419edb0 con 0x7f8ae4101060 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae419ef70 con 0x7f8ae4102250 2024-03-11T19:50:17.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4102c10 0x7f8ae419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36114/0 (socket says 172.21.15.107:36114) 2024-03-11T19:50:17.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ae2800640 1 -- 172.21.15.107:0/1361916204 learned_addr learned my addr 172.21.15.107:0/1361916204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:17.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.488+0000 7f8ad9e00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 0x7f8ae4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 -- 172.21.15.107:0/1361916204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 msgr2=0x7f8ae4197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 0x7f8ae4197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 -- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4102c10 msgr2=0x7f8ae419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4102c10 0x7f8ae419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 -- 172.21.15.107:0/1361916204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc009840 con 0x7f8ae4101060 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae2800640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4000c00 tx=0x7f8ad4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40043d0 con 0x7f8ae4101060 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad40052d0 con 0x7f8ae4101060 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.489+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad4009560 con 0x7f8ae4101060 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.490+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003ca0 con 0x7f8ae4101060 2024-03-11T19:50:17.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.490+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae419f570 con 0x7f8ae4101060 2024-03-11T19:50:17.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.491+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4108380 con 0x7f8ae4101060 2024-03-11T19:50:17.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.492+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f8ad4005440 con 0x7f8ae4101060 2024-03-11T19:50:17.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.494+0000 7f8adbe00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 0x7f8ac80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.494+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8ad40efd90 con 0x7f8ae4101060 2024-03-11T19:50:17.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.494+0000 7f8ad9e00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 0x7f8ac80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.494+0000 7f8ad9e00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 0x7f8ac80a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4198660 tx=0x7f8acc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.498+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad40b8d60 con 0x7f8ae4101060 2024-03-11T19:50:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:17 smithi107 ceph-mon[31712]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:17 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3899217519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:17 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/572271048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.695+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ae402cda0 con 0x7f8ae4101060 2024-03-11T19:50:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.696+0000 7f8adbe00640 1 -- 172.21.15.107:0/1361916204 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ad4043020 con 0x7f8ae4101060 2024-03-11T19:50:17.701 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 msgr2=0x7f8ac80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 0x7f8ac80a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4198660 tx=0x7f8acc005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 msgr2=0x7f8ae4197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4000c00 tx=0x7f8ad4005e80 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 shutdown_connections 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f8ac80a1a30 0x7f8ac80a3ef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ae4102c10 0x7f8ae419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae4102250 0x7f8ae4197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 --2- 172.21.15.107:0/1361916204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ae4101060 0x7f8ae4197150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 >> 172.21.15.107:0/1361916204 conn(0x7f8ae40fc5f0 msgr2=0x7f8ae4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 shutdown_connections 2024-03-11T19:50:17.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.699+0000 7f8ae8a00640 1 -- 172.21.15.107:0/1361916204 wait complete. 2024-03-11T19:50:17.720 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:50:17.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:17 smithi122 ceph-mon[36397]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:17 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3899217519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:17 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/572271048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:17 smithi145 ceph-mon[36730]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-11T19:50:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:17 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3899217519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:17 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/572271048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:50:17.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 -- 172.21.15.107:0/624159327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 msgr2=0x7f1fa8103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 --2- 172.21.15.107:0/624159327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8103ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009900 tx=0x7f1fa002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 -- 172.21.15.107:0/624159327 shutdown_connections 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 --2- 172.21.15.107:0/624159327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8104530 0x7f1fa8106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 --2- 172.21.15.107:0/624159327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8106e60 0x7f1fa81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 --2- 172.21.15.107:0/624159327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8103ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.911+0000 7f1faf000640 1 -- 172.21.15.107:0/624159327 >> 172.21.15.107:0/624159327 conn(0x7f1fa80fb610 msgr2=0x7f1fa80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:17.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.912+0000 7f1faf000640 1 -- 172.21.15.107:0/624159327 shutdown_connections 2024-03-11T19:50:17.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.912+0000 7f1faf000640 1 -- 172.21.15.107:0/624159327 wait complete. 2024-03-11T19:50:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.912+0000 7f1faf000640 1 Processor -- start 2024-03-11T19:50:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.912+0000 7f1faf000640 1 -- start start 2024-03-11T19:50:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 0x7f1fa80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 0x7f1fa81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8100850 con 0x7f1fa8104530 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81009c0 con 0x7f1fa8106e60 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1faf000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81a1280 con 0x7f1fa8101c00 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1fa7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 0x7f1fa80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1fad000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 0x7f1fa81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1fa7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 0x7f1fa80ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46706/0 (socket says 172.21.15.107:46706) 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1fa7400640 1 -- 172.21.15.107:0/3437785222 learned_addr learned my addr 172.21.15.107:0/3437785222 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.913+0000 7f1fa7e00640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 -- 172.21.15.107:0/3437785222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 msgr2=0x7f1fa8101570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8101570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 -- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 msgr2=0x7f1fa80ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 0x7f1fa80ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 -- 172.21.15.107:0/3437785222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009560 con 0x7f1fa8106e60 2024-03-11T19:50:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fad000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 0x7f1fa81000f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c9a0 tx=0x7f1f9800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98007bb0 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f98007d10 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa81a1530 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f980186e0 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.914+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa81a1a40 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.915+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa80fd100 con 0x7f1fa8106e60 2024-03-11T19:50:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.917+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1f980188e0 con 0x7f1fa8106e60 2024-03-11T19:50:17.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.918+0000 7f1fa4c00640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 0x7f1f8c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:17.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.918+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1f980cd520 con 0x7f1fa8106e60 2024-03-11T19:50:17.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.919+0000 7f1fa7e00640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 0x7f1f8c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:17.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.919+0000 7f1fa7e00640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 0x7f1f8c0a3f70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000c00 tx=0x7f1fa002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:17.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:17.920+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f98096510 con 0x7f1fa8106e60 2024-03-11T19:50:18.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.115+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1fa810cbe0 con 0x7f1fa8106e60 2024-03-11T19:50:18.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.116+0000 7f1fa4c00640 1 -- 172.21.15.107:0/3437785222 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1f98099bd0 con 0x7f1fa8106e60 2024-03-11T19:50:18.121 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 msgr2=0x7f1f8c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 0x7f1f8c0a3f70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000c00 tx=0x7f1fa002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 msgr2=0x7f1fa81000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 0x7f1fa81000f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c9a0 tx=0x7f1f9800ce60 comp rx=0 tx=0).stop 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 shutdown_connections 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f1f8c0a1ab0 0x7f1f8c0a3f70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fa8106e60 0x7f1fa81000f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fa8104530 0x7f1fa80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 --2- 172.21.15.107:0/3437785222 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fa8101c00 0x7f1fa8101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 >> 172.21.15.107:0/3437785222 conn(0x7f1fa80fb610 msgr2=0x7f1fa8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.118+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 shutdown_connections 2024-03-11T19:50:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.119+0000 7f1faf000640 1 -- 172.21.15.107:0/3437785222 wait complete. 2024-03-11T19:50:18.141 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:50:18.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 -- 172.21.15.107:0/892059643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4108890 msgr2=0x7fbeb4108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/892059643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4108890 0x7fbeb4108d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009900 tx=0x7fbe9c02bc40 comp rx=0 tx=0).stop 2024-03-11T19:50:18.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 -- 172.21.15.107:0/892059643 shutdown_connections 2024-03-11T19:50:18.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/892059643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4108890 0x7fbeb4108d10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/892059643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb4109250 0x7fbeb410fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/892059643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb41076a0 0x7fbeb4107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 -- 172.21.15.107:0/892059643 >> 172.21.15.107:0/892059643 conn(0x7fbeb4076650 msgr2=0x7fbeb4078a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:18.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 -- 172.21.15.107:0/892059643 shutdown_connections 2024-03-11T19:50:18.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.338+0000 7fbeb8a00640 1 -- 172.21.15.107:0/892059643 wait complete. 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.339+0000 7fbeb8a00640 1 Processor -- start 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.339+0000 7fbeb8a00640 1 -- start start 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 0x7fbeb419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4109250 0x7fbeb41a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb41a51f0 con 0x7fbeb41076a0 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb41a5360 con 0x7fbeb4109250 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb41a5520 con 0x7fbeb4108890 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46728/0 (socket says 172.21.15.107:46728) 2024-03-11T19:50:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 -- 172.21.15.107:0/307803407 learned_addr learned my addr 172.21.15.107:0/307803407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbea9e00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 0x7fbeb419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb3200640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4109250 0x7fbeb41a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 -- 172.21.15.107:0/307803407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 msgr2=0x7fbeb419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 0x7fbeb419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 -- 172.21.15.107:0/307803407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4109250 msgr2=0x7fbeb41a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4109250 0x7fbeb41a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbeb2800640 1 -- 172.21.15.107:0/307803407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c009560 con 0x7fbeb41076a0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.340+0000 7fbea9e00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 0x7fbeb419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeb2800640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbea400eb70 tx=0x7fbea400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003b70 con 0x7fbeb41076a0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea400ee70 con 0x7fbeb41076a0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40107f0 con 0x7fbeb41076a0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb41a5800 con 0x7fbeb41076a0 2024-03-11T19:50:18.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.341+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb41a5cc0 con 0x7fbeb41076a0 2024-03-11T19:50:18.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.343+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb410e960 con 0x7fbeb41076a0 2024-03-11T19:50:18.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.344+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fbea4004020 con 0x7fbeb41076a0 2024-03-11T19:50:18.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.346+0000 7fbeabe00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 0x7fbe980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:18.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.346+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbea4014070 con 0x7fbeb41076a0 2024-03-11T19:50:18.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.346+0000 7fbea9e00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 0x7fbe980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:18.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.347+0000 7fbea9e00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 0x7fbe980a3ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbeb419ec10 tx=0x7fbe9c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:18.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.351+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbea4096cc0 con 0x7fbeb41076a0 2024-03-11T19:50:18.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.564+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbeb41130b0 con 0x7fbeb41076a0 2024-03-11T19:50:18.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.564+0000 7fbeabe00640 1 -- 172.21.15.107:0/307803407 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+598 (secure 0 0 0) 0x7fbea409a380 con 0x7fbeb41076a0 2024-03-11T19:50:18.569 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1356730303 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:18.569 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:18.569 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/865925559 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/531077471 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1620686143 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2887016959 2024-03-12T19:41:54.175068+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3318828522 2024-03-12T19:41:19.200454+0000 2024-03-11T19:50:18.570 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 msgr2=0x7fbe980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 0x7fbe980a3ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbeb419ec10 tx=0x7fbe9c005be0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 msgr2=0x7fbeb419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbea400eb70 tx=0x7fbea400c6a0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 shutdown_connections 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fbe980a1a30 0x7fbe980a3ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4109250 0x7fbeb41a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fbeb4108890 0x7fbeb419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 --2- 172.21.15.107:0/307803407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeb41076a0 0x7fbeb419d700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:18.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 >> 172.21.15.107:0/307803407 conn(0x7fbeb4076650 msgr2=0x7fbeb410bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:18.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 shutdown_connections 2024-03-11T19:50:18.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:18.567+0000 7fbeb8a00640 1 -- 172.21.15.107:0/307803407 wait complete. 2024-03-11T19:50:18.592 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-03-11T19:50:18.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:18.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-11T19:50:18.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[75254]: fuse finished with error 0 and tester_r 0 2024-03-11T19:50:18.684 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T19:50:18.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:18 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1361916204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:18.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:18 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3437785222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:18.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:18 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1361916204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:18.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:18 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3437785222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:18.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:18 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1361916204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:50:18.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:18 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3437785222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:50:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:19 smithi122 ceph-mon[36397]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:19 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/307803407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:19 smithi145 ceph-mon[36730]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:19 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/307803407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:19 smithi107 ceph-mon[31712]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:19 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/307803407' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:50:21.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:21 smithi107 ceph-mon[31712]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:21.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:21 smithi122 ceph-mon[36397]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:21 smithi145 ceph-mon[36730]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:23.767 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:23 smithi122 ceph-mon[36397]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:23 smithi145 ceph-mon[36730]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:23.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:23 smithi107 ceph-mon[31712]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:24.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:50:24.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:24.786 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-11T19:50:24.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.071+0000 7f4633000640 1 -- 172.21.15.107:0/4190078597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1077c0 msgr2=0x7f462c107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.071+0000 7f4633000640 1 --2- 172.21.15.107:0/4190078597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1077c0 0x7f462c107be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f461c009900 tx=0x7f461c02bc40 comp rx=0 tx=0).stop 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 -- 172.21.15.107:0/4190078597 shutdown_connections 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 --2- 172.21.15.107:0/4190078597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c1089b0 0x7f462c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 --2- 172.21.15.107:0/4190078597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c109370 0x7f462c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 --2- 172.21.15.107:0/4190078597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1077c0 0x7f462c107be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 -- 172.21.15.107:0/4190078597 >> 172.21.15.107:0/4190078597 conn(0x7f462c076790 msgr2=0x7f462c078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 -- 172.21.15.107:0/4190078597 shutdown_connections 2024-03-11T19:50:25.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.072+0000 7f4633000640 1 -- 172.21.15.107:0/4190078597 wait complete. 2024-03-11T19:50:25.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 Processor -- start 2024-03-11T19:50:25.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 -- start start 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 0x7f462c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c109370 0x7f462c1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1a5370 con 0x7f462c1077c0 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1a54e0 con 0x7f462c109370 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.073+0000 7f4633000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1a56a0 con 0x7f462c1089b0 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51302/0 (socket says 172.21.15.107:51302) 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 -- 172.21.15.107:0/2237424182 learned_addr learned my addr 172.21.15.107:0/2237424182 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f4631000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c109370 0x7f462c1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462b400640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 0x7f462c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 -- 172.21.15.107:0/2237424182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 msgr2=0x7f462c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 0x7f462c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 -- 172.21.15.107:0/2237424182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c109370 msgr2=0x7f462c1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c109370 0x7f462c1a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 -- 172.21.15.107:0/2237424182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c009560 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462b400640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 0x7f462c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.074+0000 7f462be00640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.075+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0043d0 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.075+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f461c0052d0 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.075+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c1a5920 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.075+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003da0 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.075+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c1a5e80 con 0x7f462c1077c0 2024-03-11T19:50:25.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.076+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c10eae0 con 0x7f462c1077c0 2024-03-11T19:50:25.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.078+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f461c005440 con 0x7f462c1077c0 2024-03-11T19:50:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.079+0000 7f4628c00640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 0x7f460c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.079+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f461c0efe50 con 0x7f462c1077c0 2024-03-11T19:50:25.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.079+0000 7f462b400640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 0x7f460c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.080+0000 7f462b400640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 0x7f460c0a3f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f462c19ed90 tx=0x7f4614009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.084+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f461c0b8ee0 con 0x7f462c1077c0 2024-03-11T19:50:25.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.280+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f462c02ce40 con 0x7f462c1077c0 2024-03-11T19:50:25.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.281+0000 7f4628c00640 1 -- 172.21.15.107:0/2237424182 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f461c043070 con 0x7f462c1077c0 2024-03-11T19:50:25.287 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 msgr2=0x7f460c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 0x7f460c0a3f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f462c19ed90 tx=0x7f4614009500 comp rx=0 tx=0).stop 2024-03-11T19:50:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 msgr2=0x7f462c19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c005e80 comp rx=0 tx=0).stop 2024-03-11T19:50:25.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 shutdown_connections 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f460c0a1ab0 0x7f460c0a3f70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c109370 0x7f462c1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f462c1089b0 0x7f462c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 --2- 172.21.15.107:0/2237424182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f462c1077c0 0x7f462c19d880 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 >> 172.21.15.107:0/2237424182 conn(0x7f462c076790 msgr2=0x7f462c10bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.284+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 shutdown_connections 2024-03-11T19:50:25.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.285+0000 7f4633000640 1 -- 172.21.15.107:0/2237424182 wait complete. 2024-03-11T19:50:25.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.500+0000 7fd80ae00640 1 -- 172.21.15.107:0/1916168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 msgr2=0x7fd804104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.500+0000 7fd80ae00640 1 --2- 172.21.15.107:0/1916168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804104a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009900 tx=0x7fd7ec02bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.500+0000 7fd80ae00640 1 -- 172.21.15.107:0/1916168938 shutdown_connections 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 --2- 172.21.15.107:0/1916168938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd804104f70 0x7fd80410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 --2- 172.21.15.107:0/1916168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804104a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 --2- 172.21.15.107:0/1916168938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd8041033c0 0x7fd8041037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 -- 172.21.15.107:0/1916168938 >> 172.21.15.107:0/1916168938 conn(0x7fd8040fe970 msgr2=0x7fd804100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 -- 172.21.15.107:0/1916168938 shutdown_connections 2024-03-11T19:50:25.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.501+0000 7fd80ae00640 1 -- 172.21.15.107:0/1916168938 wait complete. 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 Processor -- start 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 -- start start 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 0x7fd804199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd804104f70 0x7fd8041a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a0f20 con 0x7fd8041045b0 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a1090 con 0x7fd8041033c0 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd80ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041a1250 con 0x7fd804104f70 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd7fbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51318/0 (socket says 172.21.15.107:51318) 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd7fbe00640 1 -- 172.21.15.107:0/2671411069 learned_addr learned my addr 172.21.15.107:0/2671411069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd803e00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 0x7fd804199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.502+0000 7fd808e00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd804104f70 0x7fd8041a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 -- 172.21.15.107:0/2671411069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd804104f70 msgr2=0x7fd8041a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd804104f70 0x7fd8041a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 -- 172.21.15.107:0/2671411069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 msgr2=0x7fd804199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 0x7fd804199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 -- 172.21.15.107:0/2671411069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4009490 con 0x7fd8041045b0 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd803e00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 0x7fd804199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd7fbe00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec02c6a0 tx=0x7fd7ec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0044c0 con 0x7fd8041045b0 2024-03-11T19:50:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0052d0 con 0x7fd8041045b0 2024-03-11T19:50:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.503+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec009560 con 0x7fd8041045b0 2024-03-11T19:50:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.504+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec003e80 con 0x7fd8041045b0 2024-03-11T19:50:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.504+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8041a1850 con 0x7fd8041045b0 2024-03-11T19:50:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.505+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80410a680 con 0x7fd8041045b0 2024-03-11T19:50:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.507+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd7ec005440 con 0x7fd8041045b0 2024-03-11T19:50:25.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.508+0000 7fd801600640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 0x7fd7e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.508+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd7ec0eff20 con 0x7fd8041045b0 2024-03-11T19:50:25.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.508+0000 7fd803e00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 0x7fd7e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.509+0000 7fd803e00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 0x7fd7e80a3ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd804104410 tx=0x7fd7f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.515+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7ec0b8fb0 con 0x7fd8041045b0 2024-03-11T19:50:25.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.713+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd80402ce40 con 0x7fd8041045b0 2024-03-11T19:50:25.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.716+0000 7fd801600640 1 -- 172.21.15.107:0/2671411069 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7fd7ec043070 con 0x7fd8041045b0 2024-03-11T19:50:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:25.721 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 msgr2=0x7fd7e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 0x7fd7e80a3ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd804104410 tx=0x7fd7f4005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 msgr2=0x7fd804199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec02c6a0 tx=0x7fd7ec0040c0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 shutdown_connections 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd7e80a1a30 0x7fd7e80a3ef0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd804104f70 0x7fd8041a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8041045b0 0x7fd804199970 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 --2- 172.21.15.107:0/2671411069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8041033c0 0x7fd804199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 >> 172.21.15.107:0/2671411069 conn(0x7fd8040fe970 msgr2=0x7fd804107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 shutdown_connections 2024-03-11T19:50:25.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.718+0000 7fd80ae00640 1 -- 172.21.15.107:0/2671411069 wait complete. 2024-03-11T19:50:25.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:25 smithi107 ceph-mon[31712]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:25.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:25 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2237424182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:25.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-11T19:50:25.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:25 smithi122 ceph-mon[36397]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:25.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:25 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2237424182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:25 smithi145 ceph-mon[36730]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:25 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2237424182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:25.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.941+0000 7f4c3ec00640 1 -- 172.21.15.107:0/2631692944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104590 msgr2=0x7f4c38104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.941+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/2631692944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104590 0x7f4c38104a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009900 tx=0x7f4c2002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 -- 172.21.15.107:0/2631692944 shutdown_connections 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/2631692944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c38104f50 0x7f4c3810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/2631692944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104590 0x7f4c38104a10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/2631692944 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c381033a0 0x7f4c381037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 -- 172.21.15.107:0/2631692944 >> 172.21.15.107:0/2631692944 conn(0x7f4c380fe930 msgr2=0x7f4c38100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 -- 172.21.15.107:0/2631692944 shutdown_connections 2024-03-11T19:50:25.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.942+0000 7f4c3ec00640 1 -- 172.21.15.107:0/2631692944 wait complete. 2024-03-11T19:50:25.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 Processor -- start 2024-03-11T19:50:25.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 -- start start 2024-03-11T19:50:25.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c381033a0 0x7f4c38199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c38104590 0x7f4c38199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a0f30 con 0x7f4c38104f50 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a10a0 con 0x7f4c381033a0 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381a1260 con 0x7f4c38104590 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51332/0 (socket says 172.21.15.107:51332) 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c3cc00640 1 -- 172.21.15.107:0/1246138125 learned_addr learned my addr 172.21.15.107:0/1246138125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c37e00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c381033a0 0x7f4c38199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.943+0000 7f4c37400640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c38104590 0x7f4c38199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 -- 172.21.15.107:0/1246138125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c38104590 msgr2=0x7f4c38199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c38104590 0x7f4c38199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 -- 172.21.15.107:0/1246138125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c381033a0 msgr2=0x7f4c38199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c381033a0 0x7f4c38199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:25.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 -- 172.21.15.107:0/1246138125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c20009560 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3cc00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000eb10 tx=0x7f4c3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30004b20 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c381a1540 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c30004c80 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30010650 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.944+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c381a19d0 con 0x7f4c38104f50 2024-03-11T19:50:25.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.945+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3810a6c0 con 0x7f4c38104f50 2024-03-11T19:50:25.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.947+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4c30010810 con 0x7f4c38104f50 2024-03-11T19:50:25.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.949+0000 7f4c34c00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 0x7f4c1c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:25.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.949+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4c30014070 con 0x7f4c38104f50 2024-03-11T19:50:25.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.949+0000 7f4c37e00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 0x7f4c1c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:25.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.950+0000 7f4c37e00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 0x7f4c1c0a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c381043f0 tx=0x7f4c28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:25.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:25.951+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c30096d70 con 0x7f4c38104f50 2024-03-11T19:50:26.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.185+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c3810edb0 con 0x7f4c38104f50 2024-03-11T19:50:26.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.520+0000 7f4c34c00640 1 -- 172.21.15.107:0/1246138125 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c3009a430 con 0x7f4c38104f50 2024-03-11T19:50:26.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 msgr2=0x7f4c1c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 0x7f4c1c0a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c381043f0 tx=0x7f4c28005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 msgr2=0x7f4c381a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000eb10 tx=0x7f4c3000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 shutdown_connections 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4c1c0a1a30 0x7f4c1c0a3ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c38104f50 0x7f4c381a09f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c38104590 0x7f4c38199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 --2- 172.21.15.107:0/1246138125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c381033a0 0x7f4c38199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 >> 172.21.15.107:0/1246138125 conn(0x7f4c380fe930 msgr2=0x7f4c38107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:26.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.526+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 shutdown_connections 2024-03-11T19:50:26.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.527+0000 7f4c3ec00640 1 -- 172.21.15.107:0/1246138125 wait complete. 2024-03-11T19:50:26.553 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-11T19:50:26.554 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 -- 172.21.15.107:0/2234442461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81033c0 msgr2=0x7f82b81037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2234442461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81033c0 0x7f82b81037e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 -- 172.21.15.107:0/2234442461 shutdown_connections 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2234442461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81045b0 0x7f82b8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2234442461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 0x7f82b810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2234442461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81033c0 0x7f82b81037e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 -- 172.21.15.107:0/2234442461 >> 172.21.15.107:0/2234442461 conn(0x7f82b80fe930 msgr2=0x7f82b8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:26.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 -- 172.21.15.107:0/2234442461 shutdown_connections 2024-03-11T19:50:26.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.765+0000 7f82bcc00640 1 -- 172.21.15.107:0/2234442461 wait complete. 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 Processor -- start 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 -- start start 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 0x7f82b81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 0x7f82b81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81a0f70 con 0x7f82b81045b0 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81a10e0 con 0x7f82b81033c0 2024-03-11T19:50:26.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.766+0000 7f82bcc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81a12a0 con 0x7f82b8104f70 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 0x7f82b81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47116/0 (socket says 172.21.15.107:47116) 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 -- 172.21.15.107:0/2344393851 learned_addr learned my addr 172.21.15.107:0/2344393851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b6800640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 0x7f82b81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 -- 172.21.15.107:0/2344393851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 msgr2=0x7f82b81a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 0x7f82b81a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 -- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 msgr2=0x7f82b81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 0x7f82b81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 -- 172.21.15.107:0/2344393851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009900 con 0x7f82b81033c0 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82ad400640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 0x7f82b81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:26.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.767+0000 7f82b5e00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82a8000f80 tx=0x7f82a8034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.768+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a80329b0 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.768+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b81a1520 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.768+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a8003e50 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.768+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8040450 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.768+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81a19f0 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.769+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b810a6e0 con 0x7f82b81033c0 2024-03-11T19:50:26.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.771+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f82a800a040 con 0x7f82b81033c0 2024-03-11T19:50:26.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.772+0000 7f82af400640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 0x7f829c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:26.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.772+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f82a80f07c0 con 0x7f82b81033c0 2024-03-11T19:50:26.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.772+0000 7f82ad400640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 0x7f829c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:26.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.773+0000 7f82ad400640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 0x7f829c0a3ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f82b819a990 tx=0x7f82a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:26.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:26.774+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f82a80b9730 con 0x7f82b81033c0 2024-03-11T19:50:26.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:26 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2671411069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:26 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:50:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:26 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2671411069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:26 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:50:27.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:26 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2671411069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:27.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:26 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:50:27.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.012+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f82b802cda0 con 0x7f82b81033c0 2024-03-11T19:50:27.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.414+0000 7f82af400640 1 -- 172.21.15.107:0/2344393851 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f82a80bcdf0 con 0x7f82b81033c0 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 msgr2=0x7f829c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 0x7f829c0a3ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f82b819a990 tx=0x7f82a0005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 msgr2=0x7f82b8199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82a8000f80 tx=0x7f82a8034a90 comp rx=0 tx=0).stop 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 shutdown_connections 2024-03-11T19:50:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f829c0a1a30 0x7f829c0a3ef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f82b8104f70 0x7f82b81a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b81045b0 0x7f82b81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 --2- 172.21.15.107:0/2344393851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82b81033c0 0x7f82b8199480 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 >> 172.21.15.107:0/2344393851 conn(0x7f82b80fe930 msgr2=0x7f82b81078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.418+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 shutdown_connections 2024-03-11T19:50:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.419+0000 7f82bcc00640 1 -- 172.21.15.107:0/2344393851 wait complete. 2024-03-11T19:50:27.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 -- 172.21.15.107:0/751315911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c109350 msgr2=0x7fe07c10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 --2- 172.21.15.107:0/751315911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c109350 0x7fe07c10fbe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe074009be0 tx=0x7fe07402bef0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 -- 172.21.15.107:0/751315911 shutdown_connections 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 --2- 172.21.15.107:0/751315911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c109350 0x7fe07c10fbe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 --2- 172.21.15.107:0/751315911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c108990 0x7fe07c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 --2- 172.21.15.107:0/751315911 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c1077a0 0x7fe07c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 -- 172.21.15.107:0/751315911 >> 172.21.15.107:0/751315911 conn(0x7fe07c076740 msgr2=0x7fe07c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:27.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 -- 172.21.15.107:0/751315911 shutdown_connections 2024-03-11T19:50:27.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.638+0000 7fe081600640 1 -- 172.21.15.107:0/751315911 wait complete. 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.639+0000 7fe081600640 1 Processor -- start 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.639+0000 7fe081600640 1 -- start start 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 0x7fe07c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 0x7fe07c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c109350 0x7fe07c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c1a5360 con 0x7fe07c109350 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c1a54d0 con 0x7fe07c1077a0 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe081600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c1a5690 con 0x7fe07c108990 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 0x7fe07c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c109350 0x7fe07c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 0x7fe07c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 0x7fe07c19ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:39274/0 (socket says 172.21.15.107:39274) 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07a000640 1 -- 172.21.15.107:0/3676862782 learned_addr learned my addr 172.21.15.107:0/3676862782 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 -- 172.21.15.107:0/3676862782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 msgr2=0x7fe07c19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 0x7fe07c19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 -- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c109350 msgr2=0x7fe07c1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c109350 0x7fe07c1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.640+0000 7fe07aa00640 1 -- 172.21.15.107:0/3676862782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009840 con 0x7fe07c1077a0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe07aa00640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 0x7fe07c19d870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005fd0 tx=0x7fe06c00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00cc90 con 0x7fe07c1077a0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe06c00ee70 con 0x7fe07c1077a0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c010430 con 0x7fe07c1077a0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c1a5970 con 0x7fe07c1077a0 2024-03-11T19:50:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.641+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c1a5e30 con 0x7fe07c1077a0 2024-03-11T19:50:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.642+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c10ea60 con 0x7fe07c1077a0 2024-03-11T19:50:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.644+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe06c003f60 con 0x7fe07c1077a0 2024-03-11T19:50:27.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.645+0000 7fe073400640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 0x7fe0600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:27.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.645+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe06c014070 con 0x7fe07c1077a0 2024-03-11T19:50:27.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.645+0000 7fe07a000640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 0x7fe0600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:27.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.646+0000 7fe07a000640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 0x7fe0600a3ef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe07c19ed80 tx=0x7fe068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:27.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.649+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe06c0968b0 con 0x7fe07c1077a0 2024-03-11T19:50:27.696 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2344393851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-11T19:50:27.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:27 smithi107 ceph-mon[31712]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-11T19:50:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2344393851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-11T19:50:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:27 smithi145 ceph-mon[36730]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-11T19:50:27.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:27.884+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe07c02ce40 con 0x7fe07c1077a0 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1246138125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2344393851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-11T19:50:28.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:27 smithi122 ceph-mon[36397]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-11T19:50:28.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.416+0000 7fe073400640 1 -- 172.21.15.107:0/3676862782 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe06c099f70 con 0x7fe07c1077a0 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 msgr2=0x7fe0600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 0x7fe0600a3ef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe07c19ed80 tx=0x7fe068005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 msgr2=0x7fe07c19d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 0x7fe07c19d870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005fd0 tx=0x7fe06c00ec20 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 shutdown_connections 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0600a1a30 0x7fe0600a3ef0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c109350 0x7fe07c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe07c108990 0x7fe07c19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 --2- 172.21.15.107:0/3676862782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe07c1077a0 0x7fe07c19d870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.420+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 >> 172.21.15.107:0/3676862782 conn(0x7fe07c076740 msgr2=0x7fe07c10bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.421+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 shutdown_connections 2024-03-11T19:50:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.421+0000 7fe081600640 1 -- 172.21.15.107:0/3676862782 wait complete. 2024-03-11T19:50:28.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-11T19:50:28.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 -- 172.21.15.107:0/3077528948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b01010e0 msgr2=0x7f87b0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 --2- 172.21.15.107:0/3077528948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b01010e0 0x7f87b0101560 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8798009890 tx=0x7f879802bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 -- 172.21.15.107:0/3077528948 shutdown_connections 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 --2- 172.21.15.107:0/3077528948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b01010e0 0x7f87b0101560 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 --2- 172.21.15.107:0/3077528948 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b0101aa0 0x7f87b01083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 --2- 172.21.15.107:0/3077528948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 -- 172.21.15.107:0/3077528948 >> 172.21.15.107:0/3077528948 conn(0x7f87b00fb4f0 msgr2=0x7f87b00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.639+0000 7f87af200640 1 -- 172.21.15.107:0/3077528948 shutdown_connections 2024-03-11T19:50:28.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.640+0000 7f87af200640 1 -- 172.21.15.107:0/3077528948 wait complete. 2024-03-11T19:50:28.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.640+0000 7f87af200640 1 Processor -- start 2024-03-11T19:50:28.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 -- start start 2024-03-11T19:50:28.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b01010e0 0x7f87b0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 0x7f87b01a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01a0e20 con 0x7f87b00fff10 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01a0f90 con 0x7f87b0101aa0 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87af200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01a1150 con 0x7f87b01010e0 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51364/0 (socket says 172.21.15.107:51364) 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87ade00640 1 -- 172.21.15.107:0/2257575892 learned_addr learned my addr 172.21.15.107:0/2257575892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87ae800640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 0x7f87b01a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.641+0000 7f87a5400640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b01010e0 0x7f87b0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 -- 172.21.15.107:0/2257575892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b01010e0 msgr2=0x7f87b0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b01010e0 0x7f87b0199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 -- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 msgr2=0x7f87b0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:28.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 -- 172.21.15.107:0/2257575892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8798009490 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ade00640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87ae800640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 0x7f87b01a08e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e970 tx=0x7f87a800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8009620 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87a800ccc0 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8004a00 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b01a1430 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.642+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b01a1910 con 0x7f87b0101aa0 2024-03-11T19:50:28.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.643+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b010b970 con 0x7f87b0101aa0 2024-03-11T19:50:28.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.647+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f87a8005020 con 0x7f87b0101aa0 2024-03-11T19:50:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.649+0000 7f87a7400640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 0x7f87940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.649+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f87a8014070 con 0x7f87b0101aa0 2024-03-11T19:50:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.649+0000 7f87ade00640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 0x7f87940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.649+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87a80964f0 con 0x7f87b0101aa0 2024-03-11T19:50:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.650+0000 7f87ade00640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 0x7f87940a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87a0003b00 tx=0x7f87a0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3676862782' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:active 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: Cluster is now healthy 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-11T19:50:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:50:28.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:28 smithi122 ceph-mon[36397]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:50:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3676862782' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:50:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:active 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: Cluster is now healthy 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:50:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:28 smithi145 ceph-mon[36730]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:50:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3676862782' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:50:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:50:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] up:active 2024-03-11T19:50:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:50:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:50:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: Cluster is now healthy 2024-03-11T19:50:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-11T19:50:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-11T19:50:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:28 smithi107 ceph-mon[31712]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-11T19:50:28.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:28.886+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f87b01a1d70 con 0x7f87b0101aa0 2024-03-11T19:50:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.431+0000 7f87a7400640 1 -- 172.21.15.107:0/2257575892 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f87a8099bb0 con 0x7f87b0101aa0 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 msgr2=0x7f87940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 0x7f87940a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87a0003b00 tx=0x7f87a0010040 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 msgr2=0x7f87b01a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 0x7f87b01a08e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e970 tx=0x7f87a800ee30 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 shutdown_connections 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f87940a1a30 0x7f87940a3ef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b0101aa0 0x7f87b01a08e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87b01010e0 0x7f87b0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 --2- 172.21.15.107:0/2257575892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b00fff10 0x7f87b0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 >> 172.21.15.107:0/2257575892 conn(0x7f87b00fb4f0 msgr2=0x7f87b00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:29.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 shutdown_connections 2024-03-11T19:50:29.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.434+0000 7f87af200640 1 -- 172.21.15.107:0/2257575892 wait complete. 2024-03-11T19:50:29.458 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-11T19:50:29.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-11T19:50:29.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 -- 172.21.15.107:0/1069110698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 msgr2=0x7fa0381010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 --2- 172.21.15.107:0/1069110698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381010d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa030009ca0 tx=0x7fa03002c190 comp rx=0 tx=0).stop 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 -- 172.21.15.107:0/1069110698 shutdown_connections 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 --2- 172.21.15.107:0/1069110698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0381002b0 0x7fa038100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 --2- 172.21.15.107:0/1069110698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381010d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 --2- 172.21.15.107:0/1069110698 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0380ff950 0x7fa0380ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.655+0000 7fa03cc00640 1 -- 172.21.15.107:0/1069110698 >> 172.21.15.107:0/1069110698 conn(0x7fa0380fb4f0 msgr2=0x7fa0380fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.656+0000 7fa03cc00640 1 -- 172.21.15.107:0/1069110698 shutdown_connections 2024-03-11T19:50:29.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.656+0000 7fa03cc00640 1 -- 172.21.15.107:0/1069110698 wait complete. 2024-03-11T19:50:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.656+0000 7fa03cc00640 1 Processor -- start 2024-03-11T19:50:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 -- start start 2024-03-11T19:50:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:29.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0381002b0 0x7fa0381997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381a0d70 con 0x7fa038100c50 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381a0ee0 con 0x7fa0380ff950 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa03cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381a10a0 con 0x7fa0381002b0 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47182/0 (socket says 172.21.15.107:47182) 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51392/0 (socket says 172.21.15.107:51392) 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037400640 1 -- 172.21.15.107:0/4012344799 learned_addr learned my addr 172.21.15.107:0/4012344799 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa036a00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0381002b0 0x7fa0381997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037400640 1 -- 172.21.15.107:0/4012344799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0381002b0 msgr2=0x7fa0381997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.657+0000 7fa037400640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0381002b0 0x7fa0381997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa037400640 1 -- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 msgr2=0x7fa0381a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa037400640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa037400640 1 -- 172.21.15.107:0/4012344799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024009490 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa037400640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa02800e9f0 tx=0x7fa02800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028009720 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa02800cd80 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028010690 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa030009900 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.658+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0381a16b0 con 0x7fa0380ff950 2024-03-11T19:50:29.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.659+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03810a6a0 con 0x7fa0380ff950 2024-03-11T19:50:29.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.661+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa0280046c0 con 0x7fa0380ff950 2024-03-11T19:50:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.662+0000 7fa02fe00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 0x7fa01c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.662+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa028014070 con 0x7fa0380ff950 2024-03-11T19:50:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.662+0000 7fa036a00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 0x7fa01c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:29.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.663+0000 7fa036a00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 0x7fa01c0a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03819a790 tx=0x7fa024005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.666+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa028096980 con 0x7fa0380ff950 2024-03-11T19:50:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2257575892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:active 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:50:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:29 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:50:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2257575892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:50:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:active 2024-03-11T19:50:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:50:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:50:29.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.882+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fa03802ce10 con 0x7fa01c0a1a30 2024-03-11T19:50:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.919+0000 7fa02fe00640 1 -- 172.21.15.107:0/4012344799 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa03802ce10 con 0x7fa01c0a1a30 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 msgr2=0x7fa01c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 0x7fa01c0a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03819a790 tx=0x7fa024005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 msgr2=0x7fa038199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa02800e9f0 tx=0x7fa02800edf0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 shutdown_connections 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa01c0a1a30 0x7fa01c0a3ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa038100c50 0x7fa0381a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0381002b0 0x7fa0381997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 --2- 172.21.15.107:0/4012344799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0380ff950 0x7fa038199280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 >> 172.21.15.107:0/4012344799 conn(0x7fa0380fb4f0 msgr2=0x7fa0380fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 shutdown_connections 2024-03-11T19:50:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:29.922+0000 7fa03cc00640 1 -- 172.21.15.107:0/4012344799 wait complete. 2024-03-11T19:50:29.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2257575892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] up:active 2024-03-11T19:50:30.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-11T19:50:30.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:29 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:50:30.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 -- 172.21.15.107:0/403410886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 msgr2=0x7fb42c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 --2- 172.21.15.107:0/403410886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c108e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb414009900 tx=0x7fb41402bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 -- 172.21.15.107:0/403410886 shutdown_connections 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 --2- 172.21.15.107:0/403410886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c109350 0x7fb42c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 --2- 172.21.15.107:0/403410886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c108e10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 --2- 172.21.15.107:0/403410886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c1077a0 0x7fb42c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 -- 172.21.15.107:0/403410886 >> 172.21.15.107:0/403410886 conn(0x7fb42c076740 msgr2=0x7fb42c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 -- 172.21.15.107:0/403410886 shutdown_connections 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.126+0000 7fb431e00640 1 -- 172.21.15.107:0/403410886 wait complete. 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 Processor -- start 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 -- start start 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c1077a0 0x7fb42c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c109350 0x7fb42c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a5390 con 0x7fb42c108990 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.127+0000 7fb431e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a5500 con 0x7fb42c1077a0 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb431e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c1a56c0 con 0x7fb42c109350 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51420/0 (socket says 172.21.15.107:51420) 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 -- 172.21.15.107:0/2108406269 learned_addr learned my addr 172.21.15.107:0/2108406269 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb42aa00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c1077a0 0x7fb42c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb42b400640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c109350 0x7fb42c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 -- 172.21.15.107:0/2108406269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c109350 msgr2=0x7fb42c1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c109350 0x7fb42c1a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 -- 172.21.15.107:0/2108406269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c1077a0 msgr2=0x7fb42c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c1077a0 0x7fb42c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.128+0000 7fb423e00640 1 -- 172.21.15.107:0/2108406269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009560 con 0x7fb42c108990 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb423e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb414006010 tx=0x7fb4140040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:30.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4140044c0 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c1a5940 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4140052d0 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c1a5ea0 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.129+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414003db0 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.130+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42c10eb90 con 0x7fb42c108990 2024-03-11T19:50:30.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.132+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fb414005440 con 0x7fb42c108990 2024-03-11T19:50:30.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.133+0000 7fb430a00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 0x7fb4100a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.133+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb4140efbf0 con 0x7fb42c108990 2024-03-11T19:50:30.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.133+0000 7fb42aa00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 0x7fb4100a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.134+0000 7fb42aa00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 0x7fb4100a3ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb42c1087f0 tx=0x7fb41c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:30.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.137+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4140b8bc0 con 0x7fb42c108990 2024-03-11T19:50:30.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:50:29.893+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:50:30.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:50:29.893+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:50:30.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:50:29.893+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:50:30.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:50:29.893+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:50:30.295 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:50:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:50:29.893+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:50:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.363+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb42c02ceb0 con 0x7fb4100a1a30 2024-03-11T19:50:30.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.394+0000 7fb430a00640 1 -- 172.21.15.107:0/2108406269 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb42c02ceb0 con 0x7fb4100a1a30 2024-03-11T19:50:30.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.396+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 msgr2=0x7fb4100a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.396+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 0x7fb4100a3ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb42c1087f0 tx=0x7fb41c005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.396+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 msgr2=0x7fb42c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.396+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb414006010 tx=0x7fb4140040c0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 shutdown_connections 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb4100a1a30 0x7fb4100a3ef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb42c109350 0x7fb42c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb42c108990 0x7fb42c19dde0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 --2- 172.21.15.107:0/2108406269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb42c1077a0 0x7fb42c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 >> 172.21.15.107:0/2108406269 conn(0x7fb42c076740 msgr2=0x7fb42c10bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 shutdown_connections 2024-03-11T19:50:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.397+0000 7fb431e00640 1 -- 172.21.15.107:0/2108406269 wait complete. 2024-03-11T19:50:30.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-11T19:50:30.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 -- 172.21.15.107:0/620594531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 msgr2=0x7f9c001015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 --2- 172.21.15.107:0/620594531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001015b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009900 tx=0x7f9bf002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 -- 172.21.15.107:0/620594531 shutdown_connections 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 --2- 172.21.15.107:0/620594531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102d40 0x7f9c001095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 --2- 172.21.15.107:0/620594531 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102380 0x7f9c00102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 --2- 172.21.15.107:0/620594531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001015b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.624+0000 7f9c08200640 1 -- 172.21.15.107:0/620594531 >> 172.21.15.107:0/620594531 conn(0x7f9c000fc720 msgr2=0x7f9c000feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.625+0000 7f9c08200640 1 -- 172.21.15.107:0/620594531 shutdown_connections 2024-03-11T19:50:30.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.625+0000 7f9c08200640 1 -- 172.21.15.107:0/620594531 wait complete. 2024-03-11T19:50:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.625+0000 7f9c08200640 1 Processor -- start 2024-03-11T19:50:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 -- start start 2024-03-11T19:50:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102380 0x7f9c00199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102d40 0x7f9c001a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001a0fb0 con 0x7f9c00101190 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001a1120 con 0x7f9c00102380 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c08200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001a12e0 con 0x7f9c00102d40 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c05800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c05800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51430/0 (socket says 172.21.15.107:51430) 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c05800640 1 -- 172.21.15.107:0/2179907652 learned_addr learned my addr 172.21.15.107:0/2179907652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c04e00640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102380 0x7f9c00199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c06200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102d40 0x7f9c001a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c05800640 1 -- 172.21.15.107:0/2179907652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102d40 msgr2=0x7f9c001a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.626+0000 7f9c05800640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102d40 0x7f9c001a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c05800640 1 -- 172.21.15.107:0/2179907652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102380 msgr2=0x7f9c00199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c05800640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102380 0x7f9c00199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c05800640 1 -- 172.21.15.107:0/2179907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009560 con 0x7f9c00101190 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c05800640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005ab0 tx=0x7f9bf0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00043d0 con 0x7f9c00101190 2024-03-11T19:50:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bf00052d0 con 0x7f9c00101190 2024-03-11T19:50:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003d40 con 0x7f9c00101190 2024-03-11T19:50:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c001a1560 con 0x7f9c00101190 2024-03-11T19:50:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.627+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c001a1a20 con 0x7f9c00101190 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.628+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0010cba0 con 0x7f9c00101190 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.630+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f9bf0005440 con 0x7f9c00101190 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.631+0000 7f9bfe000640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 0x7f9be40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.631+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9bf00ef780 con 0x7f9c00101190 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.632+0000 7f9c04e00640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 0x7f9be40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.632+0000 7f9c04e00640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 0x7f9be40a4000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0019a9d0 tx=0x7f9bec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:30.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.636+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bf00b86e0 con 0x7f9c00101190 2024-03-11T19:50:30.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:50:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: 2024-03-11T19:50:30.398+0000 7fc7e2408700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-11T19:50:30.763 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 19:50:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: 2024-03-11T19:50:30.398+0000 7fc7e2408700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-11T19:50:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:30 smithi122 ceph-mon[36397]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:30 smithi122 ceph-mon[36397]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-11T19:50:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:30 smithi122 ceph-mon[36397]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:30 smithi145 ceph-mon[36730]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:30 smithi145 ceph-mon[36730]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-11T19:50:30.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:30 smithi145 ceph-mon[36730]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:30 smithi107 ceph-mon[31712]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:30 smithi107 ceph-mon[31712]: pgmap v338: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-03-11T19:50:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:30 smithi107 ceph-mon[31712]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:30.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.845+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c00074ad0 con 0x7f9be40a1b40 2024-03-11T19:50:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.854+0000 7f9bfe000640 1 -- 172.21.15.107:0/2179907652 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f9c00074ad0 con 0x7f9be40a1b40 2024-03-11T19:50:30.859 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_0/724c7a8a-fd3c-4644-85be-c5470be4c82f 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 msgr2=0x7f9be40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 0x7f9be40a4000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0019a9d0 tx=0x7f9bec005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 msgr2=0x7f9c001994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005ab0 tx=0x7f9bf0005e50 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 shutdown_connections 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f9be40a1b40 0x7f9be40a4000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00102380 0x7f9c00199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00102d40 0x7f9c001a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 --2- 172.21.15.107:0/2179907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00101190 0x7f9c001994c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 >> 172.21.15.107:0/2179907652 conn(0x7f9c000fc720 msgr2=0x7f9c000733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 shutdown_connections 2024-03-11T19:50:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:30.857+0000 7f9c08200640 1 -- 172.21.15.107:0/2179907652 wait complete. 2024-03-11T19:50:30.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-11T19:50:31.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 -- 172.21.15.107:0/815456431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36381033a0 msgr2=0x7f36381037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 --2- 172.21.15.107:0/815456431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36381033a0 0x7f36381037c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f362c009900 tx=0x7f362c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:31.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 -- 172.21.15.107:0/815456431 shutdown_connections 2024-03-11T19:50:31.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 --2- 172.21.15.107:0/815456431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3638104590 0x7f3638104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 --2- 172.21.15.107:0/815456431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 0x7f363810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 --2- 172.21.15.107:0/815456431 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36381033a0 0x7f36381037c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 -- 172.21.15.107:0/815456431 >> 172.21.15.107:0/815456431 conn(0x7f36380fe930 msgr2=0x7f3638100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:31.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 -- 172.21.15.107:0/815456431 shutdown_connections 2024-03-11T19:50:31.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.077+0000 7f363fe00640 1 -- 172.21.15.107:0/815456431 wait complete. 2024-03-11T19:50:31.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.078+0000 7f363fe00640 1 Processor -- start 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.078+0000 7f363fe00640 1 -- start start 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3638104590 0x7f3638199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 0x7f36381a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381a0f40 con 0x7f3638104f50 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381a10b0 con 0x7f36381033a0 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381a1270 con 0x7f3638104590 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 0x7f36381a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47252/0 (socket says 172.21.15.107:47252) 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 -- 172.21.15.107:0/3253686553 learned_addr learned my addr 172.21.15.107:0/3253686553 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363ca00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3638104590 0x7f3638199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 -- 172.21.15.107:0/3253686553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3638104590 msgr2=0x7f3638199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3638104590 0x7f3638199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 -- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 msgr2=0x7f36381a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 0x7f36381a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.079+0000 7f363d400640 1 -- 172.21.15.107:0/3253686553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c009560 con 0x7f36381033a0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f363d400640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f362c005ab0 tx=0x7f362c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c004aa0 con 0x7f36381033a0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36381a14f0 con 0x7f36381033a0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f362c0052d0 con 0x7f36381033a0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c003e80 con 0x7f36381033a0 2024-03-11T19:50:31.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.080+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36381a1a50 con 0x7f36381033a0 2024-03-11T19:50:31.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.081+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363810a740 con 0x7f36381033a0 2024-03-11T19:50:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.083+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f362c005440 con 0x7f36381033a0 2024-03-11T19:50:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.084+0000 7f3636000640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 0x7f36200a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.085+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f362c0f0120 con 0x7f36381033a0 2024-03-11T19:50:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.085+0000 7f363ca00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 0x7f36200a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.085+0000 7f363ca00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 0x7f36200a3f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f363819a960 tx=0x7f3624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:31.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.087+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f362c0b9040 con 0x7f36381033a0 2024-03-11T19:50:31.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.297+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f363802ce40 con 0x7f36200a1ac0 2024-03-11T19:50:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.327+0000 7f3636000640 1 -- 172.21.15.107:0/3253686553 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f363802ce40 con 0x7f36200a1ac0 2024-03-11T19:50:31.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 msgr2=0x7f36200a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 0x7f36200a3f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f363819a960 tx=0x7f3624005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 msgr2=0x7f3638199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f362c005ab0 tx=0x7f362c004270 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 shutdown_connections 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.329+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f36200a1ac0 0x7f36200a3f80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3638104f50 0x7f36381a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3638104590 0x7f3638199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 --2- 172.21.15.107:0/3253686553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36381033a0 0x7f3638199450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 >> 172.21.15.107:0/3253686553 conn(0x7f36380fe930 msgr2=0x7f3638107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 shutdown_connections 2024-03-11T19:50:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.330+0000 7f363fe00640 1 -- 172.21.15.107:0/3253686553 wait complete. 2024-03-11T19:50:31.352 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 -- 172.21.15.107:0/3140676983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104f50 msgr2=0x7f55a010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 --2- 172.21.15.107:0/3140676983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104f50 0x7f55a010b7e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ca0 tx=0x7f559802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 -- 172.21.15.107:0/3140676983 shutdown_connections 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 --2- 172.21.15.107:0/3140676983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a0104590 0x7f55a0104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 --2- 172.21.15.107:0/3140676983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104f50 0x7f55a010b7e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 --2- 172.21.15.107:0/3140676983 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a01033a0 0x7f55a01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 -- 172.21.15.107:0/3140676983 >> 172.21.15.107:0/3140676983 conn(0x7f55a00fe930 msgr2=0x7f55a0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 -- 172.21.15.107:0/3140676983 shutdown_connections 2024-03-11T19:50:31.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.560+0000 7f55a6c00640 1 -- 172.21.15.107:0/3140676983 wait complete. 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.561+0000 7f55a6c00640 1 Processor -- start 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.561+0000 7f55a6c00640 1 -- start start 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.561+0000 7f55a6c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 0x7f55a01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.561+0000 7f55a6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f55a6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a0104f50 0x7f55a01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f55a6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a01a0ee0 con 0x7f55a0104590 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f55a6c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a01a1050 con 0x7f55a01033a0 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f55a6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a01a1210 con 0x7f55a0104f50 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51478/0 (socket says 172.21.15.107:51478) 2024-03-11T19:50:31.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 -- 172.21.15.107:0/1300080846 learned_addr learned my addr 172.21.15.107:0/1300080846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f55a4c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a0104f50 0x7f55a01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f559fe00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 0x7f55a01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 -- 172.21.15.107:0/1300080846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a0104f50 msgr2=0x7f55a01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a0104f50 0x7f55a01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 -- 172.21.15.107:0/1300080846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 msgr2=0x7f55a01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 0x7f55a01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f5597e00640 1 -- 172.21.15.107:0/1300080846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598009900 con 0x7f55a0104590 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.562+0000 7f559fe00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 0x7f55a01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f5597e00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588009700 con 0x7f55a0104590 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f558800cd80 con 0x7f55a0104590 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a01a14f0 con 0x7f55a0104590 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588018920 con 0x7f55a0104590 2024-03-11T19:50:31.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.563+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a01a1a50 con 0x7f55a0104590 2024-03-11T19:50:31.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.564+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a010a660 con 0x7f55a0104590 2024-03-11T19:50:31.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.566+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f5588018a80 con 0x7f55a0104590 2024-03-11T19:50:31.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.567+0000 7f559d600640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 0x7f55840a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:31.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.567+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5588014070 con 0x7f55a0104590 2024-03-11T19:50:31.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.567+0000 7f559fe00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 0x7f55840a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:31.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.568+0000 7f559fe00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 0x7f55840a4000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55a01043f0 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:31.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.571+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5588052d20 con 0x7f55a0104590 2024-03-11T19:50:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:31 smithi145 ceph-mon[36730]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:50:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:31 smithi145 ceph-mon[36730]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:31 smithi145 ceph-mon[36730]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:31 smithi107 ceph-mon[31712]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:50:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:31 smithi107 ceph-mon[31712]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:31 smithi107 ceph-mon[31712]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.797+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a002ceb0 con 0x7f55840a1b40 2024-03-11T19:50:31.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.802+0000 7f559d600640 1 -- 172.21.15.107:0/1300080846 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f55a002ceb0 con 0x7f55840a1b40 2024-03-11T19:50:31.807 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_1/1159a975-ef10-422d-92cf-d6ce759e4d5b 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 msgr2=0x7f55840a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 0x7f55840a4000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55a01043f0 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 msgr2=0x7f55a0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f558800e9f0 tx=0x7f558800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 shutdown_connections 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f55840a1b40 0x7f55840a4000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f55a0104f50 0x7f55a01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a0104590 0x7f55a0199930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 --2- 172.21.15.107:0/1300080846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55a01033a0 0x7f55a01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:31.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 >> 172.21.15.107:0/1300080846 conn(0x7f55a00fe930 msgr2=0x7f55a0107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:31.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 shutdown_connections 2024-03-11T19:50:31.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:31.805+0000 7f55a6c00640 1 -- 172.21.15.107:0/1300080846 wait complete. 2024-03-11T19:50:31.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:50:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:31 smithi122 ceph-mon[36397]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-11T19:50:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:31 smithi122 ceph-mon[36397]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:31 smithi122 ceph-mon[36397]: from='client.24854 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 -- 172.21.15.107:0/2889290488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c109370 msgr2=0x7fee4c10fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 --2- 172.21.15.107:0/2889290488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c109370 0x7fee4c10fc00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee44009be0 tx=0x7fee4402bef0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 -- 172.21.15.107:0/2889290488 shutdown_connections 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 --2- 172.21.15.107:0/2889290488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c109370 0x7fee4c10fc00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 --2- 172.21.15.107:0/2889290488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c1089b0 0x7fee4c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 --2- 172.21.15.107:0/2889290488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1077c0 0x7fee4c107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 -- 172.21.15.107:0/2889290488 >> 172.21.15.107:0/2889290488 conn(0x7fee4c076790 msgr2=0x7fee4c078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 -- 172.21.15.107:0/2889290488 shutdown_connections 2024-03-11T19:50:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.026+0000 7fee54000640 1 -- 172.21.15.107:0/2889290488 wait complete. 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 Processor -- start 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 -- start start 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c1077c0 0x7fee4c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c109370 0x7fee4c1a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a52e0 con 0x7fee4c1089b0 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a5450 con 0x7fee4c1077c0 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.027+0000 7fee54000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a5610 con 0x7fee4c109370 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51498/0 (socket says 172.21.15.107:51498) 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 -- 172.21.15.107:0/3792896253 learned_addr learned my addr 172.21.15.107:0/3792896253 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee52000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c109370 0x7fee4c1a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee51600640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c1077c0 0x7fee4c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 -- 172.21.15.107:0/3792896253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c109370 msgr2=0x7fee4c1a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c109370 0x7fee4c1a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 -- 172.21.15.107:0/3792896253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c1077c0 msgr2=0x7fee4c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c1077c0 0x7fee4c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 -- 172.21.15.107:0/3792896253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009560 con 0x7fee4c1089b0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.028+0000 7fee50c00640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e9f0 tx=0x7fee3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.029+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c009720 con 0x7fee4c1089b0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.029+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee3c00cd80 con 0x7fee4c1089b0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.029+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee44009840 con 0x7fee4c1089b0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.029+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c010690 con 0x7fee4c1089b0 2024-03-11T19:50:32.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.029+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1a5c20 con 0x7fee4c1089b0 2024-03-11T19:50:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.030+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c10eae0 con 0x7fee4c1089b0 2024-03-11T19:50:32.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.032+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fee3c0046c0 con 0x7fee4c1089b0 2024-03-11T19:50:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.034+0000 7fee4a000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 0x7fee340a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.034+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fee3c014070 con 0x7fee4c1089b0 2024-03-11T19:50:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.034+0000 7fee51600640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 0x7fee340a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.035+0000 7fee51600640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 0x7fee340a3f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee40009530 tx=0x7fee40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.038+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee3c0969b0 con 0x7fee4c1089b0 2024-03-11T19:50:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.259+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee4c1131c0 con 0x7fee4c1089b0 2024-03-11T19:50:32.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.261+0000 7fee4a000640 1 -- 172.21.15.107:0/3792896253 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fee3c09a070 con 0x7fee4c1089b0 2024-03-11T19:50:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:32.266 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:50:32.267 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:50:32.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 msgr2=0x7fee340a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 0x7fee340a3f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee40009530 tx=0x7fee40009260 comp rx=0 tx=0).stop 2024-03-11T19:50:32.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 msgr2=0x7fee4c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e9f0 tx=0x7fee3c00edf0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 shutdown_connections 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fee340a1ac0 0x7fee340a3f80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fee4c109370 0x7fee4c1a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4c1089b0 0x7fee4c19dd30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 --2- 172.21.15.107:0/3792896253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4c1077c0 0x7fee4c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 >> 172.21.15.107:0/3792896253 conn(0x7fee4c076790 msgr2=0x7fee4c10bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 shutdown_connections 2024-03-11T19:50:32.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.264+0000 7fee54000640 1 -- 172.21.15.107:0/3792896253 wait complete. 2024-03-11T19:50:32.286 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:50:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 -- 172.21.15.107:0/470047326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 msgr2=0x7f231c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 --2- 172.21.15.107:0/470047326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1037e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f230c009860 tx=0x7f230c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 -- 172.21.15.107:0/470047326 shutdown_connections 2024-03-11T19:50:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 --2- 172.21.15.107:0/470047326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c104f70 0x7f231c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 --2- 172.21.15.107:0/470047326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c1045b0 0x7f231c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 --2- 172.21.15.107:0/470047326 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1037e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 -- 172.21.15.107:0/470047326 >> 172.21.15.107:0/470047326 conn(0x7f231c0fe930 msgr2=0x7f231c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 -- 172.21.15.107:0/470047326 shutdown_connections 2024-03-11T19:50:32.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.476+0000 7f2321000640 1 -- 172.21.15.107:0/470047326 wait complete. 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.477+0000 7f2321000640 1 Processor -- start 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 -- start start 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 0x7f231c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c104f70 0x7f231c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1a0fa0 con 0x7f231c104f70 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1a1110 con 0x7f231c1045b0 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2321000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1a12d0 con 0x7f231c1033c0 2024-03-11T19:50:32.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2319e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2319e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:39430/0 (socket says 172.21.15.107:39430) 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f2319e00640 1 -- 172.21.15.107:0/1562714380 learned_addr learned my addr 172.21.15.107:0/1562714380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.478+0000 7f231a800640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c104f70 0x7f231c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2313e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 0x7f231c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 -- 172.21.15.107:0/1562714380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 msgr2=0x7f231c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 0x7f231c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 -- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c104f70 msgr2=0x7f231c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c104f70 0x7f231c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 -- 172.21.15.107:0/1562714380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314009840 con 0x7f231c1033c0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2313e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 0x7f231c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2319e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f230c02c0b0 tx=0x7f230c034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c02c510 con 0x7f231c1033c0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c009490 con 0x7f231c1033c0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f230c02cb50 con 0x7f231c1033c0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c032860 con 0x7f231c1033c0 2024-03-11T19:50:32.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.479+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c1a1730 con 0x7f231c1033c0 2024-03-11T19:50:32.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.480+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c10a780 con 0x7f231c1033c0 2024-03-11T19:50:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.483+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f230c03b040 con 0x7f231c1033c0 2024-03-11T19:50:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.484+0000 7f2312a00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 0x7f23000a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.484+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f230c0ef750 con 0x7f231c1033c0 2024-03-11T19:50:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.484+0000 7f2313e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 0x7f23000a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.485+0000 7f2313e00640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 0x7f23000a3f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.486+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f230c0b86d0 con 0x7f231c1033c0 2024-03-11T19:50:32.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.686+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f231c02cdd0 con 0x7f231c1033c0 2024-03-11T19:50:32.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.688+0000 7f2312a00640 1 -- 172.21.15.107:0/1562714380 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7f230c0bbd90 con 0x7f231c1033c0 2024-03-11T19:50:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:32.693 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 msgr2=0x7f23000a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 0x7f23000a3f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2304005fd0 tx=0x7f2304005a20 comp rx=0 tx=0).stop 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 msgr2=0x7f231c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f230c02c0b0 tx=0x7f230c034440 comp rx=0 tx=0).stop 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 shutdown_connections 2024-03-11T19:50:32.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23000a1ac0 0x7f23000a3f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231c1045b0 0x7f231c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f231c104f70 0x7f231c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 --2- 172.21.15.107:0/1562714380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f231c1033c0 0x7f231c1994b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.691+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 >> 172.21.15.107:0/1562714380 conn(0x7f231c0fe930 msgr2=0x7f231c107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.692+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 shutdown_connections 2024-03-11T19:50:32.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.692+0000 7f2321000640 1 -- 172.21.15.107:0/1562714380 wait complete. 2024-03-11T19:50:32.716 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-11T19:50:32.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 -- 172.21.15.107:0/998881264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938106e80 msgr2=0x7f49381092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 --2- 172.21.15.107:0/998881264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938106e80 0x7f49381092d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f493002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 -- 172.21.15.107:0/998881264 shutdown_connections 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 --2- 172.21.15.107:0/998881264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938106e80 0x7f49381092d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 --2- 172.21.15.107:0/998881264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938104550 0x7f4938106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 --2- 172.21.15.107:0/998881264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938101c20 0x7f4938104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 -- 172.21.15.107:0/998881264 >> 172.21.15.107:0/998881264 conn(0x7f49380fb610 msgr2=0x7f49380fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.912+0000 7f493d400640 1 -- 172.21.15.107:0/998881264 shutdown_connections 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.913+0000 7f493d400640 1 -- 172.21.15.107:0/998881264 wait complete. 2024-03-11T19:50:32.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.913+0000 7f493d400640 1 Processor -- start 2024-03-11T19:50:32.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 -- start start 2024-03-11T19:50:32.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938101c20 0x7f4938197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 0x7f493819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819ec70 con 0x7f4938106e80 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819ede0 con 0x7f4938104550 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f493d400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493819efa0 con 0x7f4938101c20 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f492de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f4937200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 0x7f493819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f492de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47306/0 (socket says 172.21.15.107:47306) 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f4937200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 0x7f493819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51540/0 (socket says 172.21.15.107:51540) 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f4936800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938101c20 0x7f4938197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.914+0000 7f492de00640 1 -- 172.21.15.107:0/2115748602 learned_addr learned my addr 172.21.15.107:0/2115748602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 -- 172.21.15.107:0/2115748602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938101c20 msgr2=0x7f4938197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938101c20 0x7f4938197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 -- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 msgr2=0x7f493819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 0x7f493819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 -- 172.21.15.107:0/2115748602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009560 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492de00640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4920005fd0 tx=0x7f492000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920009c30 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4920009d90 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920010460 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493819f280 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.915+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493819f740 con 0x7f4938104550 2024-03-11T19:50:32.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.917+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49380fd100 con 0x7f4938104550 2024-03-11T19:50:32.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.919+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f4920003f20 con 0x7f4938104550 2024-03-11T19:50:32.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.920+0000 7f492fe00640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 0x7f491c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:32.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.920+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49200cdb30 con 0x7f4938104550 2024-03-11T19:50:32.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.920+0000 7f4936800640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 0x7f491c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:32.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.921+0000 7f4936800640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 0x7f491c0a4000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4928007ec0 tx=0x7f49280075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:32.922+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49200969c0 con 0x7f4938104550 2024-03-11T19:50:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:32 smithi122 ceph-mon[36397]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:32 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3792896253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:32 smithi122 ceph-mon[36397]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-11T19:50:33.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:32 smithi145 ceph-mon[36730]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:33.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:32 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3792896253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:32 smithi145 ceph-mon[36730]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-11T19:50:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:32 smithi107 ceph-mon[31712]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:50:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:32 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3792896253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:32 smithi107 ceph-mon[31712]: pgmap v339: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-11T19:50:33.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.116+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f493802cf80 con 0x7f491c0a1b40 2024-03-11T19:50:33.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.133+0000 7f492fe00640 1 -- 172.21.15.107:0/2115748602 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340583 (secure 0 0 0) 0x7f493802cf80 con 0x7f491c0a1b40 2024-03-11T19:50:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:33.145 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:50:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 msgr2=0x7f491c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 0x7f491c0a4000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4928007ec0 tx=0x7f49280075b0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 msgr2=0x7f49381976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4920005fd0 tx=0x7f492000dde0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 shutdown_connections 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f491c0a1b40 0x7f491c0a4000 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4938104550 0x7f49381976c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4938106e80 0x7f493819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 --2- 172.21.15.107:0/2115748602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4938101c20 0x7f4938197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.140+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 >> 172.21.15.107:0/2115748602 conn(0x7f49380fb610 msgr2=0x7f4938102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.141+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 shutdown_connections 2024-03-11T19:50:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.141+0000 7f493d400640 1 -- 172.21.15.107:0/2115748602 wait complete. 2024-03-11T19:50:33.163 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-03-11T19:50:32.330717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597730,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":151,"num_read_kb":133,"num_write":304,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352640,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589184,"statfs":{"total":1151940427776,"available":1151579324416,"internally_reserved":0,"allocated":12079104,"data_stored":7635341,"data_compressed":283668,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2863,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":15,"num_read_kb":14,"num_write":75,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006630"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526589+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:50:02.526589+0000","last_peered":"2024-03-11T19:50:02.526589+0000","last_clean":"2024-03-11T19:50:02.526589+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:50:02.526589+0000","last_undegraded":"2024-03-11T19:50:02.526589+0000","last_fullsized":"2024-03-11T19:50:02.526589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526960+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:50:02.526960+0000","last_peered":"2024-03-11T19:50:02.526960+0000","last_clean":"2024-03-11T19:50:02.526960+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:50:02.526960+0000","last_undegraded":"2024-03-11T19:50:02.526960+0000","last_fullsized":"2024-03-11T19:50:02.526960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525943+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:50:02.525943+0000","last_peered":"2024-03-11T19:50:02.525943+0000","last_clean":"2024-03-11T19:50:02.525943+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:50:02.525943+0000","last_undegraded":"2024-03-11T19:50:02.525943+0000","last_fullsized":"2024-03-11T19:50:02.525943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526149+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:50:02.526149+0000","last_peered":"2024-03-11T19:50:02.526149+0000","last_clean":"2024-03-11T19:50:02.526149+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:50:02.526149+0000","last_undegraded":"2024-03-11T19:50:02.526149+0000","last_fullsized":"2024-03-11T19:50:02.526149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525858+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:50:02.525858+0000","last_peered":"2024-03-11T19:50:02.525858+0000","last_clean":"2024-03-11T19:50:02.525858+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:50:02.525858+0000","last_undegraded":"2024-03-11T19:50:02.525858+0000","last_fullsized":"2024-03-11T19:50:02.525858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526245+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:50:02.526245+0000","last_peered":"2024-03-11T19:50:02.526245+0000","last_clean":"2024-03-11T19:50:02.526245+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:50:02.526245+0000","last_undegraded":"2024-03-11T19:50:02.526245+0000","last_fullsized":"2024-03-11T19:50:02.526245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526904+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:50:02.526904+0000","last_peered":"2024-03-11T19:50:02.526904+0000","last_clean":"2024-03-11T19:50:02.526904+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:50:02.526904+0000","last_undegraded":"2024-03-11T19:50:02.526904+0000","last_fullsized":"2024-03-11T19:50:02.526904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.577655+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:50:01.577655+0000","last_peered":"2024-03-11T19:50:01.577655+0000","last_clean":"2024-03-11T19:50:01.577655+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:50:01.577655+0000","last_undegraded":"2024-03-11T19:50:01.577655+0000","last_fullsized":"2024-03-11T19:50:01.577655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429582+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:50:01.429582+0000","last_peered":"2024-03-11T19:50:01.429582+0000","last_clean":"2024-03-11T19:50:01.429582+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:50:01.429582+0000","last_undegraded":"2024-03-11T19:50:01.429582+0000","last_fullsized":"2024-03-11T19:50:01.429582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526775+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:50:02.526775+0000","last_peered":"2024-03-11T19:50:02.526775+0000","last_clean":"2024-03-11T19:50:02.526775+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:50:02.526775+0000","last_undegraded":"2024-03-11T19:50:02.526775+0000","last_fullsized":"2024-03-11T19:50:02.526775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527359+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:50:02.527359+0000","last_peered":"2024-03-11T19:50:02.527359+0000","last_clean":"2024-03-11T19:50:02.527359+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:50:02.527359+0000","last_undegraded":"2024-03-11T19:50:02.527359+0000","last_fullsized":"2024-03-11T19:50:02.527359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525717+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:50:02.525717+0000","last_peered":"2024-03-11T19:50:02.525717+0000","last_clean":"2024-03-11T19:50:02.525717+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:50:02.525717+0000","last_undegraded":"2024-03-11T19:50:02.525717+0000","last_fullsized":"2024-03-11T19:50:02.525717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430776+0000","last_change":"2024-03-11T19:49:55.201525+0000","last_active":"2024-03-11T19:50:01.430776+0000","last_peered":"2024-03-11T19:50:01.430776+0000","last_clean":"2024-03-11T19:50:01.430776+0000","last_became_active":"2024-03-11T19:49:55.199729+0000","last_became_peered":"2024-03-11T19:49:55.199729+0000","last_unstale":"2024-03-11T19:50:01.430776+0000","last_undegraded":"2024-03-11T19:50:01.430776+0000","last_fullsized":"2024-03-11T19:50:01.430776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:03:26.753822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525435+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:50:02.525435+0000","last_peered":"2024-03-11T19:50:02.525435+0000","last_clean":"2024-03-11T19:50:02.525435+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:50:02.525435+0000","last_undegraded":"2024-03-11T19:50:02.525435+0000","last_fullsized":"2024-03-11T19:50:02.525435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525313+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:50:02.525313+0000","last_peered":"2024-03-11T19:50:02.525313+0000","last_clean":"2024-03-11T19:50:02.525313+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:50:02.525313+0000","last_undegraded":"2024-03-11T19:50:02.525313+0000","last_fullsized":"2024-03-11T19:50:02.525313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.428918+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:50:01.428918+0000","last_peered":"2024-03-11T19:50:01.428918+0000","last_clean":"2024-03-11T19:50:01.428918+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:50:01.428918+0000","last_undegraded":"2024-03-11T19:50:01.428918+0000","last_fullsized":"2024-03-11T19:50:01.428918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526488+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:50:02.526488+0000","last_peered":"2024-03-11T19:50:02.526488+0000","last_clean":"2024-03-11T19:50:02.526488+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:50:02.526488+0000","last_undegraded":"2024-03-11T19:50:02.526488+0000","last_fullsized":"2024-03-11T19:50:02.526488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.528750+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:50:02.528750+0000","last_peered":"2024-03-11T19:50:02.528750+0000","last_clean":"2024-03-11T19:50:02.528750+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:50:02.528750+0000","last_undegraded":"2024-03-11T19:50:02.528750+0000","last_fullsized":"2024-03-11T19:50:02.528750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527178+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:50:02.527178+0000","last_peered":"2024-03-11T19:50:02.527178+0000","last_clean":"2024-03-11T19:50:02.527178+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:50:02.527178+0000","last_undegraded":"2024-03-11T19:50:02.527178+0000","last_fullsized":"2024-03-11T19:50:02.527178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526791+0000","last_change":"2024-03-11T19:50:00.466474+0000","last_active":"2024-03-11T19:50:02.526791+0000","last_peered":"2024-03-11T19:50:02.526791+0000","last_clean":"2024-03-11T19:50:02.526791+0000","last_became_active":"2024-03-11T19:50:00.466158+0000","last_became_peered":"2024-03-11T19:50:00.466158+0000","last_unstale":"2024-03-11T19:50:02.526791+0000","last_undegraded":"2024-03-11T19:50:02.526791+0000","last_fullsized":"2024-03-11T19:50:02.526791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:12:32.502464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.442652+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:50:28.442652+0000","last_peered":"2024-03-11T19:50:28.442652+0000","last_clean":"2024-03-11T19:50:28.442652+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:50:28.442652+0000","last_undegraded":"2024-03-11T19:50:28.442652+0000","last_fullsized":"2024-03-11T19:50:28.442652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578521+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:50:01.578521+0000","last_peered":"2024-03-11T19:50:01.578521+0000","last_clean":"2024-03-11T19:50:01.578521+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:50:01.578521+0000","last_undegraded":"2024-03-11T19:50:01.578521+0000","last_fullsized":"2024-03-11T19:50:01.578521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525318+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:50:02.525318+0000","last_peered":"2024-03-11T19:50:02.525318+0000","last_clean":"2024-03-11T19:50:02.525318+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:50:02.525318+0000","last_undegraded":"2024-03-11T19:50:02.525318+0000","last_fullsized":"2024-03-11T19:50:02.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.579057+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:50:01.579057+0000","last_peered":"2024-03-11T19:50:01.579057+0000","last_clean":"2024-03-11T19:50:01.579057+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:50:01.579057+0000","last_undegraded":"2024-03-11T19:50:01.579057+0000","last_fullsized":"2024-03-11T19:50:01.579057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526705+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:50:02.526705+0000","last_peered":"2024-03-11T19:50:02.526705+0000","last_clean":"2024-03-11T19:50:02.526705+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:50:02.526705+0000","last_undegraded":"2024-03-11T19:50:02.526705+0000","last_fullsized":"2024-03-11T19:50:02.526705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526309+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:50:02.526309+0000","last_peered":"2024-03-11T19:50:02.526309+0000","last_clean":"2024-03-11T19:50:02.526309+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:50:02.526309+0000","last_undegraded":"2024-03-11T19:50:02.526309+0000","last_fullsized":"2024-03-11T19:50:02.526309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.435657+0000","last_change":"2024-03-11T19:49:54.709796+0000","last_active":"2024-03-11T19:50:28.435657+0000","last_peered":"2024-03-11T19:50:28.435657+0000","last_clean":"2024-03-11T19:50:28.435657+0000","last_became_active":"2024-03-11T19:49:54.709073+0000","last_became_peered":"2024-03-11T19:49:54.709073+0000","last_unstale":"2024-03-11T19:50:28.435657+0000","last_undegraded":"2024-03-11T19:50:28.435657+0000","last_fullsized":"2024-03-11T19:50:28.435657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:33.720661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429349+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:50:01.429349+0000","last_peered":"2024-03-11T19:50:01.429349+0000","last_clean":"2024-03-11T19:50:01.429349+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:50:01.429349+0000","last_undegraded":"2024-03-11T19:50:01.429349+0000","last_fullsized":"2024-03-11T19:50:01.429349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525449+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:50:02.525449+0000","last_peered":"2024-03-11T19:50:02.525449+0000","last_clean":"2024-03-11T19:50:02.525449+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:50:02.525449+0000","last_undegraded":"2024-03-11T19:50:02.525449+0000","last_fullsized":"2024-03-11T19:50:02.525449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526320+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:50:02.526320+0000","last_peered":"2024-03-11T19:50:02.526320+0000","last_clean":"2024-03-11T19:50:02.526320+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:50:02.526320+0000","last_undegraded":"2024-03-11T19:50:02.526320+0000","last_fullsized":"2024-03-11T19:50:02.526320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431661+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:50:01.431661+0000","last_peered":"2024-03-11T19:50:01.431661+0000","last_clean":"2024-03-11T19:50:01.431661+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:50:01.431661+0000","last_undegraded":"2024-03-11T19:50:01.431661+0000","last_fullsized":"2024-03-11T19:50:01.431661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526978+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:50:02.526978+0000","last_peered":"2024-03-11T19:50:02.526978+0000","last_clean":"2024-03-11T19:50:02.526978+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:50:02.526978+0000","last_undegraded":"2024-03-11T19:50:02.526978+0000","last_fullsized":"2024-03-11T19:50:02.526978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.428178+0000","last_change":"2024-03-11T19:49:54.717213+0000","last_active":"2024-03-11T19:50:27.428178+0000","last_peered":"2024-03-11T19:50:27.428178+0000","last_clean":"2024-03-11T19:50:27.428178+0000","last_became_active":"2024-03-11T19:49:54.716898+0000","last_became_peered":"2024-03-11T19:49:54.716898+0000","last_unstale":"2024-03-11T19:50:27.428178+0000","last_undegraded":"2024-03-11T19:50:27.428178+0000","last_fullsized":"2024-03-11T19:50:27.428178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:33.468535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526053+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:50:02.526053+0000","last_peered":"2024-03-11T19:50:02.526053+0000","last_clean":"2024-03-11T19:50:02.526053+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:50:02.526053+0000","last_undegraded":"2024-03-11T19:50:02.526053+0000","last_fullsized":"2024-03-11T19:50:02.526053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525659+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:50:02.525659+0000","last_peered":"2024-03-11T19:50:02.525659+0000","last_clean":"2024-03-11T19:50:02.525659+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:50:02.525659+0000","last_undegraded":"2024-03-11T19:50:02.525659+0000","last_fullsized":"2024-03-11T19:50:02.525659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526198+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:50:02.526198+0000","last_peered":"2024-03-11T19:50:02.526198+0000","last_clean":"2024-03-11T19:50:02.526198+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:50:02.526198+0000","last_undegraded":"2024-03-11T19:50:02.526198+0000","last_fullsized":"2024-03-11T19:50:02.526198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431825+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:50:01.431825+0000","last_peered":"2024-03-11T19:50:01.431825+0000","last_clean":"2024-03-11T19:50:01.431825+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:50:01.431825+0000","last_undegraded":"2024-03-11T19:50:01.431825+0000","last_fullsized":"2024-03-11T19:50:01.431825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526560+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:50:02.526560+0000","last_peered":"2024-03-11T19:50:02.526560+0000","last_clean":"2024-03-11T19:50:02.526560+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:50:02.526560+0000","last_undegraded":"2024-03-11T19:50:02.526560+0000","last_fullsized":"2024-03-11T19:50:02.526560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.440392+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:50:28.440392+0000","last_peered":"2024-03-11T19:50:28.440392+0000","last_clean":"2024-03-11T19:50:28.440392+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:50:28.440392+0000","last_undegraded":"2024-03-11T19:50:28.440392+0000","last_fullsized":"2024-03-11T19:50:28.440392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.577887+0000","last_change":"2024-03-11T19:49:55.237576+0000","last_active":"2024-03-11T19:50:01.577887+0000","last_peered":"2024-03-11T19:50:01.577887+0000","last_clean":"2024-03-11T19:50:01.577887+0000","last_became_active":"2024-03-11T19:49:55.236259+0000","last_became_peered":"2024-03-11T19:49:55.236259+0000","last_unstale":"2024-03-11T19:50:01.577887+0000","last_undegraded":"2024-03-11T19:50:01.577887+0000","last_fullsized":"2024-03-11T19:50:01.577887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:05.057879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526631+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:50:02.526631+0000","last_peered":"2024-03-11T19:50:02.526631+0000","last_clean":"2024-03-11T19:50:02.526631+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:50:02.526631+0000","last_undegraded":"2024-03-11T19:50:02.526631+0000","last_fullsized":"2024-03-11T19:50:02.526631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431242+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:50:01.431242+0000","last_peered":"2024-03-11T19:50:01.431242+0000","last_clean":"2024-03-11T19:50:01.431242+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:50:01.431242+0000","last_undegraded":"2024-03-11T19:50:01.431242+0000","last_fullsized":"2024-03-11T19:50:01.431242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527456+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:50:02.527456+0000","last_peered":"2024-03-11T19:50:02.527456+0000","last_clean":"2024-03-11T19:50:02.527456+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:50:02.527456+0000","last_undegraded":"2024-03-11T19:50:02.527456+0000","last_fullsized":"2024-03-11T19:50:02.527456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578051+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:50:01.578051+0000","last_peered":"2024-03-11T19:50:01.578051+0000","last_clean":"2024-03-11T19:50:01.578051+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:50:01.578051+0000","last_undegraded":"2024-03-11T19:50:01.578051+0000","last_fullsized":"2024-03-11T19:50:01.578051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.427778+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:50:27.427778+0000","last_peered":"2024-03-11T19:50:27.427778+0000","last_clean":"2024-03-11T19:50:27.427778+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:50:27.427778+0000","last_undegraded":"2024-03-11T19:50:27.427778+0000","last_fullsized":"2024-03-11T19:50:27.427778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429560+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:50:01.429560+0000","last_peered":"2024-03-11T19:50:01.429560+0000","last_clean":"2024-03-11T19:50:01.429560+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:50:01.429560+0000","last_undegraded":"2024-03-11T19:50:01.429560+0000","last_fullsized":"2024-03-11T19:50:01.429560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527065+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:50:02.527065+0000","last_peered":"2024-03-11T19:50:02.527065+0000","last_clean":"2024-03-11T19:50:02.527065+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:50:02.527065+0000","last_undegraded":"2024-03-11T19:50:02.527065+0000","last_fullsized":"2024-03-11T19:50:02.527065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.433576+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:50:27.433576+0000","last_peered":"2024-03-11T19:50:27.433576+0000","last_clean":"2024-03-11T19:50:27.433576+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:50:27.433576+0000","last_undegraded":"2024-03-11T19:50:27.433576+0000","last_fullsized":"2024-03-11T19:50:27.433576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526646+0000","last_change":"2024-03-11T19:49:55.221391+0000","last_active":"2024-03-11T19:50:02.526646+0000","last_peered":"2024-03-11T19:50:02.526646+0000","last_clean":"2024-03-11T19:50:02.526646+0000","last_became_active":"2024-03-11T19:49:55.212401+0000","last_became_peered":"2024-03-11T19:49:55.212401+0000","last_unstale":"2024-03-11T19:50:02.526646+0000","last_undegraded":"2024-03-11T19:50:02.526646+0000","last_fullsized":"2024-03-11T19:50:02.526646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:29:20.005530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525780+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:50:02.525780+0000","last_peered":"2024-03-11T19:50:02.525780+0000","last_clean":"2024-03-11T19:50:02.525780+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:50:02.525780+0000","last_undegraded":"2024-03-11T19:50:02.525780+0000","last_fullsized":"2024-03-11T19:50:02.525780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431859+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:50:01.431859+0000","last_peered":"2024-03-11T19:50:01.431859+0000","last_clean":"2024-03-11T19:50:01.431859+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:50:01.431859+0000","last_undegraded":"2024-03-11T19:50:01.431859+0000","last_fullsized":"2024-03-11T19:50:01.431859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429539+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:50:01.429539+0000","last_peered":"2024-03-11T19:50:01.429539+0000","last_clean":"2024-03-11T19:50:01.429539+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:50:01.429539+0000","last_undegraded":"2024-03-11T19:50:01.429539+0000","last_fullsized":"2024-03-11T19:50:01.429539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527757+0000","last_change":"2024-03-11T19:50:00.467047+0000","last_active":"2024-03-11T19:50:02.527757+0000","last_peered":"2024-03-11T19:50:02.527757+0000","last_clean":"2024-03-11T19:50:02.527757+0000","last_became_active":"2024-03-11T19:50:00.466637+0000","last_became_peered":"2024-03-11T19:50:00.466637+0000","last_unstale":"2024-03-11T19:50:02.527757+0000","last_undegraded":"2024-03-11T19:50:02.527757+0000","last_fullsized":"2024-03-11T19:50:02.527757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:42:00.183583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.426019+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:50:27.426019+0000","last_peered":"2024-03-11T19:50:27.426019+0000","last_clean":"2024-03-11T19:50:27.426019+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:50:27.426019+0000","last_undegraded":"2024-03-11T19:50:27.426019+0000","last_fullsized":"2024-03-11T19:50:27.426019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526855+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:50:02.526855+0000","last_peered":"2024-03-11T19:50:02.526855+0000","last_clean":"2024-03-11T19:50:02.526855+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:50:02.526855+0000","last_undegraded":"2024-03-11T19:50:02.526855+0000","last_fullsized":"2024-03-11T19:50:02.526855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.432219+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:50:01.432219+0000","last_peered":"2024-03-11T19:50:01.432219+0000","last_clean":"2024-03-11T19:50:01.432219+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:50:01.432219+0000","last_undegraded":"2024-03-11T19:50:01.432219+0000","last_fullsized":"2024-03-11T19:50:01.432219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.436620+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:50:28.436620+0000","last_peered":"2024-03-11T19:50:28.436620+0000","last_clean":"2024-03-11T19:50:28.436620+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:50:28.436620+0000","last_undegraded":"2024-03-11T19:50:28.436620+0000","last_fullsized":"2024-03-11T19:50:28.436620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430613+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:50:01.430613+0000","last_peered":"2024-03-11T19:50:01.430613+0000","last_clean":"2024-03-11T19:50:01.430613+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:50:01.430613+0000","last_undegraded":"2024-03-11T19:50:01.430613+0000","last_fullsized":"2024-03-11T19:50:01.430613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525716+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:50:02.525716+0000","last_peered":"2024-03-11T19:50:02.525716+0000","last_clean":"2024-03-11T19:50:02.525716+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:50:02.525716+0000","last_undegraded":"2024-03-11T19:50:02.525716+0000","last_fullsized":"2024-03-11T19:50:02.525716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.429530+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:50:27.429530+0000","last_peered":"2024-03-11T19:50:27.429530+0000","last_clean":"2024-03-11T19:50:27.429530+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:50:27.429530+0000","last_undegraded":"2024-03-11T19:50:27.429530+0000","last_fullsized":"2024-03-11T19:50:27.429530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527048+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:50:02.527048+0000","last_peered":"2024-03-11T19:50:02.527048+0000","last_clean":"2024-03-11T19:50:02.527048+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:50:02.527048+0000","last_undegraded":"2024-03-11T19:50:02.527048+0000","last_fullsized":"2024-03-11T19:50:02.527048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527952+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:50:02.527952+0000","last_peered":"2024-03-11T19:50:02.527952+0000","last_clean":"2024-03-11T19:50:02.527952+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:50:02.527952+0000","last_undegraded":"2024-03-11T19:50:02.527952+0000","last_fullsized":"2024-03-11T19:50:02.527952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:29.447885+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:50:29.447885+0000","last_peered":"2024-03-11T19:50:29.447885+0000","last_clean":"2024-03-11T19:50:29.447885+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:50:29.447885+0000","last_undegraded":"2024-03-11T19:50:29.447885+0000","last_fullsized":"2024-03-11T19:50:29.447885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526383+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:50:02.526383+0000","last_peered":"2024-03-11T19:50:02.526383+0000","last_clean":"2024-03-11T19:50:02.526383+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:50:02.526383+0000","last_undegraded":"2024-03-11T19:50:02.526383+0000","last_fullsized":"2024-03-11T19:50:02.526383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431739+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:50:01.431739+0000","last_peered":"2024-03-11T19:50:01.431739+0000","last_clean":"2024-03-11T19:50:01.431739+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:50:01.431739+0000","last_undegraded":"2024-03-11T19:50:01.431739+0000","last_fullsized":"2024-03-11T19:50:01.431739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.439341+0000","last_change":"2024-03-11T19:49:54.708749+0000","last_active":"2024-03-11T19:50:28.439341+0000","last_peered":"2024-03-11T19:50:28.439341+0000","last_clean":"2024-03-11T19:50:28.439341+0000","last_became_active":"2024-03-11T19:49:54.708246+0000","last_became_peered":"2024-03-11T19:49:54.708246+0000","last_unstale":"2024-03-11T19:50:28.439341+0000","last_undegraded":"2024-03-11T19:50:28.439341+0000","last_fullsized":"2024-03-11T19:50:28.439341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:50.434110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431402+0000","last_change":"2024-03-11T19:49:55.214263+0000","last_active":"2024-03-11T19:50:01.431402+0000","last_peered":"2024-03-11T19:50:01.431402+0000","last_clean":"2024-03-11T19:50:01.431402+0000","last_became_active":"2024-03-11T19:49:55.213940+0000","last_became_peered":"2024-03-11T19:49:55.213940+0000","last_unstale":"2024-03-11T19:50:01.431402+0000","last_undegraded":"2024-03-11T19:50:01.431402+0000","last_fullsized":"2024-03-11T19:50:01.431402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:28.733906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431572+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:50:01.431572+0000","last_peered":"2024-03-11T19:50:01.431572+0000","last_clean":"2024-03-11T19:50:01.431572+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:50:01.431572+0000","last_undegraded":"2024-03-11T19:50:01.431572+0000","last_fullsized":"2024-03-11T19:50:01.431572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526735+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:50:02.526735+0000","last_peered":"2024-03-11T19:50:02.526735+0000","last_clean":"2024-03-11T19:50:02.526735+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:50:02.526735+0000","last_undegraded":"2024-03-11T19:50:02.526735+0000","last_fullsized":"2024-03-11T19:50:02.526735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526213+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:50:02.526213+0000","last_peered":"2024-03-11T19:50:02.526213+0000","last_clean":"2024-03-11T19:50:02.526213+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:50:02.526213+0000","last_undegraded":"2024-03-11T19:50:02.526213+0000","last_fullsized":"2024-03-11T19:50:02.526213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525906+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:50:02.525906+0000","last_peered":"2024-03-11T19:50:02.525906+0000","last_clean":"2024-03-11T19:50:02.525906+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:50:02.525906+0000","last_undegraded":"2024-03-11T19:50:02.525906+0000","last_fullsized":"2024-03-11T19:50:02.525906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.528380+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:50:02.528380+0000","last_peered":"2024-03-11T19:50:02.528380+0000","last_clean":"2024-03-11T19:50:02.528380+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:50:02.528380+0000","last_undegraded":"2024-03-11T19:50:02.528380+0000","last_fullsized":"2024-03-11T19:50:02.528380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526636+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:50:02.526636+0000","last_peered":"2024-03-11T19:50:02.526636+0000","last_clean":"2024-03-11T19:50:02.526636+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:50:02.526636+0000","last_undegraded":"2024-03-11T19:50:02.526636+0000","last_fullsized":"2024-03-11T19:50:02.526636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430960+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:50:01.430960+0000","last_peered":"2024-03-11T19:50:01.430960+0000","last_clean":"2024-03-11T19:50:01.430960+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:50:01.430960+0000","last_undegraded":"2024-03-11T19:50:01.430960+0000","last_fullsized":"2024-03-11T19:50:01.430960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527254+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:50:02.527254+0000","last_peered":"2024-03-11T19:50:02.527254+0000","last_clean":"2024-03-11T19:50:02.527254+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:50:02.527254+0000","last_undegraded":"2024-03-11T19:50:02.527254+0000","last_fullsized":"2024-03-11T19:50:02.527254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526366+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:50:02.526366+0000","last_peered":"2024-03-11T19:50:02.526366+0000","last_clean":"2024-03-11T19:50:02.526366+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:50:02.526366+0000","last_undegraded":"2024-03-11T19:50:02.526366+0000","last_fullsized":"2024-03-11T19:50:02.526366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431624+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:50:01.431624+0000","last_peered":"2024-03-11T19:50:01.431624+0000","last_clean":"2024-03-11T19:50:01.431624+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:50:01.431624+0000","last_undegraded":"2024-03-11T19:50:01.431624+0000","last_fullsized":"2024-03-11T19:50:01.431624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431371+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:50:01.431371+0000","last_peered":"2024-03-11T19:50:01.431371+0000","last_clean":"2024-03-11T19:50:01.431371+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:50:01.431371+0000","last_undegraded":"2024-03-11T19:50:01.431371+0000","last_fullsized":"2024-03-11T19:50:01.431371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.438920+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:50:28.438920+0000","last_peered":"2024-03-11T19:50:28.438920+0000","last_clean":"2024-03-11T19:50:28.438920+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:50:28.438920+0000","last_undegraded":"2024-03-11T19:50:28.438920+0000","last_fullsized":"2024-03-11T19:50:28.438920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527139+0000","last_change":"2024-03-11T19:50:00.464425+0000","last_active":"2024-03-11T19:50:02.527139+0000","last_peered":"2024-03-11T19:50:02.527139+0000","last_clean":"2024-03-11T19:50:02.527139+0000","last_became_active":"2024-03-11T19:50:00.463990+0000","last_became_peered":"2024-03-11T19:50:00.463990+0000","last_unstale":"2024-03-11T19:50:02.527139+0000","last_undegraded":"2024-03-11T19:50:02.527139+0000","last_fullsized":"2024-03-11T19:50:02.527139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:28:29.975190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578192+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:50:01.578192+0000","last_peered":"2024-03-11T19:50:01.578192+0000","last_clean":"2024-03-11T19:50:01.578192+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:50:01.578192+0000","last_undegraded":"2024-03-11T19:50:01.578192+0000","last_fullsized":"2024-03-11T19:50:01.578192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:29.445439+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:50:29.445439+0000","last_peered":"2024-03-11T19:50:29.445439+0000","last_clean":"2024-03-11T19:50:29.445439+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:50:29.445439+0000","last_undegraded":"2024-03-11T19:50:29.445439+0000","last_fullsized":"2024-03-11T19:50:29.445439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526176+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:50:02.526176+0000","last_peered":"2024-03-11T19:50:02.526176+0000","last_clean":"2024-03-11T19:50:02.526176+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:50:02.526176+0000","last_undegraded":"2024-03-11T19:50:02.526176+0000","last_fullsized":"2024-03-11T19:50:02.526176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.432019+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:50:01.432019+0000","last_peered":"2024-03-11T19:50:01.432019+0000","last_clean":"2024-03-11T19:50:01.432019+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:50:01.432019+0000","last_undegraded":"2024-03-11T19:50:01.432019+0000","last_fullsized":"2024-03-11T19:50:01.432019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430005+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:50:01.430005+0000","last_peered":"2024-03-11T19:50:01.430005+0000","last_clean":"2024-03-11T19:50:01.430005+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:50:01.430005+0000","last_undegraded":"2024-03-11T19:50:01.430005+0000","last_fullsized":"2024-03-11T19:50:01.430005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.435599+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:50:28.435599+0000","last_peered":"2024-03-11T19:50:28.435599+0000","last_clean":"2024-03-11T19:50:28.435599+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:50:28.435599+0000","last_undegraded":"2024-03-11T19:50:28.435599+0000","last_fullsized":"2024-03-11T19:50:28.435599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525648+0000","last_change":"2024-03-11T19:50:00.468421+0000","last_active":"2024-03-11T19:50:02.525648+0000","last_peered":"2024-03-11T19:50:02.525648+0000","last_clean":"2024-03-11T19:50:02.525648+0000","last_became_active":"2024-03-11T19:50:00.467973+0000","last_became_peered":"2024-03-11T19:50:00.467973+0000","last_unstale":"2024-03-11T19:50:02.525648+0000","last_undegraded":"2024-03-11T19:50:02.525648+0000","last_fullsized":"2024-03-11T19:50:02.525648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:50:56.817000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.431634+0000","last_change":"2024-03-11T19:49:54.720835+0000","last_active":"2024-03-11T19:50:27.431634+0000","last_peered":"2024-03-11T19:50:27.431634+0000","last_clean":"2024-03-11T19:50:27.431634+0000","last_became_active":"2024-03-11T19:49:54.720436+0000","last_became_peered":"2024-03-11T19:49:54.720436+0000","last_unstale":"2024-03-11T19:50:27.431634+0000","last_undegraded":"2024-03-11T19:50:27.431634+0000","last_fullsized":"2024-03-11T19:50:27.431634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:20:08.706822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431566+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:50:01.431566+0000","last_peered":"2024-03-11T19:50:01.431566+0000","last_clean":"2024-03-11T19:50:01.431566+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:50:01.431566+0000","last_undegraded":"2024-03-11T19:50:01.431566+0000","last_fullsized":"2024-03-11T19:50:01.431566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578063+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:50:01.578063+0000","last_peered":"2024-03-11T19:50:01.578063+0000","last_clean":"2024-03-11T19:50:01.578063+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:50:01.578063+0000","last_undegraded":"2024-03-11T19:50:01.578063+0000","last_fullsized":"2024-03-11T19:50:01.578063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.441762+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:50:28.441762+0000","last_peered":"2024-03-11T19:50:28.441762+0000","last_clean":"2024-03-11T19:50:28.441762+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:50:28.441762+0000","last_undegraded":"2024-03-11T19:50:28.441762+0000","last_fullsized":"2024-03-11T19:50:28.441762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578935+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:50:01.578935+0000","last_peered":"2024-03-11T19:50:01.578935+0000","last_clean":"2024-03-11T19:50:01.578935+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:50:01.578935+0000","last_undegraded":"2024-03-11T19:50:01.578935+0000","last_fullsized":"2024-03-11T19:50:01.578935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526534+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:50:02.526534+0000","last_peered":"2024-03-11T19:50:02.526534+0000","last_clean":"2024-03-11T19:50:02.526534+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:50:02.526534+0000","last_undegraded":"2024-03-11T19:50:02.526534+0000","last_fullsized":"2024-03-11T19:50:02.526534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.425913+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:50:27.425913+0000","last_peered":"2024-03-11T19:50:27.425913+0000","last_clean":"2024-03-11T19:50:27.425913+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:50:27.425913+0000","last_undegraded":"2024-03-11T19:50:27.425913+0000","last_fullsized":"2024-03-11T19:50:27.425913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527349+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:50:02.527349+0000","last_peered":"2024-03-11T19:50:02.527349+0000","last_clean":"2024-03-11T19:50:02.527349+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:50:02.527349+0000","last_undegraded":"2024-03-11T19:50:02.527349+0000","last_fullsized":"2024-03-11T19:50:02.527349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525774+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:50:02.525774+0000","last_peered":"2024-03-11T19:50:02.525774+0000","last_clean":"2024-03-11T19:50:02.525774+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:50:02.525774+0000","last_undegraded":"2024-03-11T19:50:02.525774+0000","last_fullsized":"2024-03-11T19:50:02.525774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.433707+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:50:27.433707+0000","last_peered":"2024-03-11T19:50:27.433707+0000","last_clean":"2024-03-11T19:50:27.433707+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:50:27.433707+0000","last_undegraded":"2024-03-11T19:50:27.433707+0000","last_fullsized":"2024-03-11T19:50:27.433707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525586+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:50:02.525586+0000","last_peered":"2024-03-11T19:50:02.525586+0000","last_clean":"2024-03-11T19:50:02.525586+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:50:02.525586+0000","last_undegraded":"2024-03-11T19:50:02.525586+0000","last_fullsized":"2024-03-11T19:50:02.525586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.529111+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:50:02.529111+0000","last_peered":"2024-03-11T19:50:02.529111+0000","last_clean":"2024-03-11T19:50:02.529111+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:50:02.529111+0000","last_undegraded":"2024-03-11T19:50:02.529111+0000","last_fullsized":"2024-03-11T19:50:02.529111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578122+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:50:01.578122+0000","last_peered":"2024-03-11T19:50:01.578122+0000","last_clean":"2024-03-11T19:50:01.578122+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:50:01.578122+0000","last_undegraded":"2024-03-11T19:50:01.578122+0000","last_fullsized":"2024-03-11T19:50:01.578122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525225+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:50:02.525225+0000","last_peered":"2024-03-11T19:50:02.525225+0000","last_clean":"2024-03-11T19:50:02.525225+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:50:02.525225+0000","last_undegraded":"2024-03-11T19:50:02.525225+0000","last_fullsized":"2024-03-11T19:50:02.525225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526509+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:50:02.526509+0000","last_peered":"2024-03-11T19:50:02.526509+0000","last_clean":"2024-03-11T19:50:02.526509+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:50:02.526509+0000","last_undegraded":"2024-03-11T19:50:02.526509+0000","last_fullsized":"2024-03-11T19:50:02.526509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525522+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:50:02.525522+0000","last_peered":"2024-03-11T19:50:02.525522+0000","last_clean":"2024-03-11T19:50:02.525522+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:50:02.525522+0000","last_undegraded":"2024-03-11T19:50:02.525522+0000","last_fullsized":"2024-03-11T19:50:02.525522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578297+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:50:01.578297+0000","last_peered":"2024-03-11T19:50:01.578297+0000","last_clean":"2024-03-11T19:50:01.578297+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:50:01.578297+0000","last_undegraded":"2024-03-11T19:50:01.578297+0000","last_fullsized":"2024-03-11T19:50:01.578297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525710+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:50:02.525710+0000","last_peered":"2024-03-11T19:50:02.525710+0000","last_clean":"2024-03-11T19:50:02.525710+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:50:02.525710+0000","last_undegraded":"2024-03-11T19:50:02.525710+0000","last_fullsized":"2024-03-11T19:50:02.525710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.428801+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:50:27.428801+0000","last_peered":"2024-03-11T19:50:27.428801+0000","last_clean":"2024-03-11T19:50:27.428801+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:50:27.428801+0000","last_undegraded":"2024-03-11T19:50:27.428801+0000","last_fullsized":"2024-03-11T19:50:27.428801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527075+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:50:02.527075+0000","last_peered":"2024-03-11T19:50:02.527075+0000","last_clean":"2024-03-11T19:50:02.527075+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:50:02.527075+0000","last_undegraded":"2024-03-11T19:50:02.527075+0000","last_fullsized":"2024-03-11T19:50:02.527075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429162+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:50:01.429162+0000","last_peered":"2024-03-11T19:50:01.429162+0000","last_clean":"2024-03-11T19:50:01.429162+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:50:01.429162+0000","last_undegraded":"2024-03-11T19:50:01.429162+0000","last_fullsized":"2024-03-11T19:50:01.429162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.438714+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:50:28.438714+0000","last_peered":"2024-03-11T19:50:28.438714+0000","last_clean":"2024-03-11T19:50:28.438714+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:50:28.438714+0000","last_undegraded":"2024-03-11T19:50:28.438714+0000","last_fullsized":"2024-03-11T19:50:28.438714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527391+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:50:02.527391+0000","last_peered":"2024-03-11T19:50:02.527391+0000","last_clean":"2024-03-11T19:50:02.527391+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:50:02.527391+0000","last_undegraded":"2024-03-11T19:50:02.527391+0000","last_fullsized":"2024-03-11T19:50:02.527391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525736+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:50:02.525736+0000","last_peered":"2024-03-11T19:50:02.525736+0000","last_clean":"2024-03-11T19:50:02.525736+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:50:02.525736+0000","last_undegraded":"2024-03-11T19:50:02.525736+0000","last_fullsized":"2024-03-11T19:50:02.525736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431951+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:50:01.431951+0000","last_peered":"2024-03-11T19:50:01.431951+0000","last_clean":"2024-03-11T19:50:01.431951+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:50:01.431951+0000","last_undegraded":"2024-03-11T19:50:01.431951+0000","last_fullsized":"2024-03-11T19:50:01.431951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527094+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:50:02.527094+0000","last_peered":"2024-03-11T19:50:02.527094+0000","last_clean":"2024-03-11T19:50:02.527094+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:50:02.527094+0000","last_undegraded":"2024-03-11T19:50:02.527094+0000","last_fullsized":"2024-03-11T19:50:02.527094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431228+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:50:01.431228+0000","last_peered":"2024-03-11T19:50:01.431228+0000","last_clean":"2024-03-11T19:50:01.431228+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:50:01.431228+0000","last_undegraded":"2024-03-11T19:50:01.431228+0000","last_fullsized":"2024-03-11T19:50:01.431228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431436+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:50:01.431436+0000","last_peered":"2024-03-11T19:50:01.431436+0000","last_clean":"2024-03-11T19:50:01.431436+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:50:01.431436+0000","last_undegraded":"2024-03-11T19:50:01.431436+0000","last_fullsized":"2024-03-11T19:50:01.431436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527535+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:50:02.527535+0000","last_peered":"2024-03-11T19:50:02.527535+0000","last_clean":"2024-03-11T19:50:02.527535+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:50:02.527535+0000","last_undegraded":"2024-03-11T19:50:02.527535+0000","last_fullsized":"2024-03-11T19:50:02.527535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525321+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:50:02.525321+0000","last_peered":"2024-03-11T19:50:02.525321+0000","last_clean":"2024-03-11T19:50:02.525321+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:50:02.525321+0000","last_undegraded":"2024-03-11T19:50:02.525321+0000","last_fullsized":"2024-03-11T19:50:02.525321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526673+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:50:02.526673+0000","last_peered":"2024-03-11T19:50:02.526673+0000","last_clean":"2024-03-11T19:50:02.526673+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:50:02.526673+0000","last_undegraded":"2024-03-11T19:50:02.526673+0000","last_fullsized":"2024-03-11T19:50:02.526673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525343+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:50:02.525343+0000","last_peered":"2024-03-11T19:50:02.525343+0000","last_clean":"2024-03-11T19:50:02.525343+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:50:02.525343+0000","last_undegraded":"2024-03-11T19:50:02.525343+0000","last_fullsized":"2024-03-11T19:50:02.525343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526053+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:50:02.526053+0000","last_peered":"2024-03-11T19:50:02.526053+0000","last_clean":"2024-03-11T19:50:02.526053+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:50:02.526053+0000","last_undegraded":"2024-03-11T19:50:02.526053+0000","last_fullsized":"2024-03-11T19:50:02.526053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431576+0000","last_change":"2024-03-11T19:49:55.218937+0000","last_active":"2024-03-11T19:50:01.431576+0000","last_peered":"2024-03-11T19:50:01.431576+0000","last_clean":"2024-03-11T19:50:01.431576+0000","last_became_active":"2024-03-11T19:49:55.218621+0000","last_became_peered":"2024-03-11T19:49:55.218621+0000","last_unstale":"2024-03-11T19:50:01.431576+0000","last_undegraded":"2024-03-11T19:50:01.431576+0000","last_fullsized":"2024-03-11T19:50:01.431576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:56.928038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526257+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:50:02.526257+0000","last_peered":"2024-03-11T19:50:02.526257+0000","last_clean":"2024-03-11T19:50:02.526257+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:50:02.526257+0000","last_undegraded":"2024-03-11T19:50:02.526257+0000","last_fullsized":"2024-03-11T19:50:02.526257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526160+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:50:02.526160+0000","last_peered":"2024-03-11T19:50:02.526160+0000","last_clean":"2024-03-11T19:50:02.526160+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:50:02.526160+0000","last_undegraded":"2024-03-11T19:50:02.526160+0000","last_fullsized":"2024-03-11T19:50:02.526160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.579077+0000","last_change":"2024-03-11T19:49:55.232791+0000","last_active":"2024-03-11T19:50:01.579077+0000","last_peered":"2024-03-11T19:50:01.579077+0000","last_clean":"2024-03-11T19:50:01.579077+0000","last_became_active":"2024-03-11T19:49:55.229966+0000","last_became_peered":"2024-03-11T19:49:55.229966+0000","last_unstale":"2024-03-11T19:50:01.579077+0000","last_undegraded":"2024-03-11T19:50:01.579077+0000","last_fullsized":"2024-03-11T19:50:01.579077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:17.049558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431305+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:50:01.431305+0000","last_peered":"2024-03-11T19:50:01.431305+0000","last_clean":"2024-03-11T19:50:01.431305+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:50:01.431305+0000","last_undegraded":"2024-03-11T19:50:01.431305+0000","last_fullsized":"2024-03-11T19:50:01.431305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578308+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:50:01.578308+0000","last_peered":"2024-03-11T19:50:01.578308+0000","last_clean":"2024-03-11T19:50:01.578308+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:50:01.578308+0000","last_undegraded":"2024-03-11T19:50:01.578308+0000","last_fullsized":"2024-03-11T19:50:01.578308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429233+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:50:01.429233+0000","last_peered":"2024-03-11T19:50:01.429233+0000","last_clean":"2024-03-11T19:50:01.429233+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:50:01.429233+0000","last_undegraded":"2024-03-11T19:50:01.429233+0000","last_fullsized":"2024-03-11T19:50:01.429233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430857+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:50:01.430857+0000","last_peered":"2024-03-11T19:50:01.430857+0000","last_clean":"2024-03-11T19:50:01.430857+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:50:01.430857+0000","last_undegraded":"2024-03-11T19:50:01.430857+0000","last_fullsized":"2024-03-11T19:50:01.430857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525859+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:50:02.525859+0000","last_peered":"2024-03-11T19:50:02.525859+0000","last_clean":"2024-03-11T19:50:02.525859+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:50:02.525859+0000","last_undegraded":"2024-03-11T19:50:02.525859+0000","last_fullsized":"2024-03-11T19:50:02.525859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7362,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":15,"num_read_kb":14,"num_write":123,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":65374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":485394,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1079798,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486212,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486132,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1077554,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485484,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":508535,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486872,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490586,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":485498,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486038,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1077238,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":12183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:50:33.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 -- 172.21.15.107:0/875437167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41045b0 msgr2=0x7fa7d4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 --2- 172.21.15.107:0/875437167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41045b0 0x7fa7d4104a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009900 tx=0x7fa7c002bc40 comp rx=0 tx=0).stop 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 -- 172.21.15.107:0/875437167 shutdown_connections 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 --2- 172.21.15.107:0/875437167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 0x7fa7d410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 --2- 172.21.15.107:0/875437167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41045b0 0x7fa7d4104a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 --2- 172.21.15.107:0/875437167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41033c0 0x7fa7d41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 -- 172.21.15.107:0/875437167 >> 172.21.15.107:0/875437167 conn(0x7fa7d40fe930 msgr2=0x7fa7d4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 -- 172.21.15.107:0/875437167 shutdown_connections 2024-03-11T19:50:33.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.412+0000 7fa7d9000640 1 -- 172.21.15.107:0/875437167 wait complete. 2024-03-11T19:50:33.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 Processor -- start 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 -- start start 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 0x7fa7d4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 0x7fa7d41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41a0f40 con 0x7fa7d41045b0 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41a10b0 con 0x7fa7d4104f70 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d9000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41a1270 con 0x7fa7d41033c0 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51564/0 (socket says 172.21.15.107:51564) 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.413+0000 7fa7d1400640 1 -- 172.21.15.107:0/2408040995 learned_addr learned my addr 172.21.15.107:0/2408040995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d2800640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 0x7fa7d41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1e00640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 0x7fa7d4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 -- 172.21.15.107:0/2408040995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 msgr2=0x7fa7d4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 0x7fa7d4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 -- 172.21.15.107:0/2408040995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 msgr2=0x7fa7d41a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 0x7fa7d41a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 -- 172.21.15.107:0/2408040995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0009560 con 0x7fa7d41045b0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1e00640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 0x7fa7d4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d1400640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c002c6b0 tx=0x7fa7c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00044e0 con 0x7fa7d41045b0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.414+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d41a14f0 con 0x7fa7d41045b0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.415+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d41a1a50 con 0x7fa7d41045b0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.415+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7c0004ac0 con 0x7fa7d41045b0 2024-03-11T19:50:33.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.415+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0003df0 con 0x7fa7d41045b0 2024-03-11T19:50:33.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.416+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b40052c0 con 0x7fa7d41045b0 2024-03-11T19:50:33.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.417+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fa7c00035f0 con 0x7fa7d41045b0 2024-03-11T19:50:33.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.419+0000 7fa7caa00640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 0x7fa7b80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.419+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa7c00ef980 con 0x7fa7d41045b0 2024-03-11T19:50:33.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.419+0000 7fa7d1e00640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 0x7fa7b80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.420+0000 7fa7d1e00640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 0x7fa7b80a3f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4104410 tx=0x7fa7c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.423+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7c00b8900 con 0x7fa7d41045b0 2024-03-11T19:50:33.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.605+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7b4000bf0 con 0x7fa7b80a1ac0 2024-03-11T19:50:33.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.630+0000 7fa7caa00640 1 -- 172.21.15.107:0/2408040995 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340583 (secure 0 0 0) 0x7fa7b4000bf0 con 0x7fa7b80a1ac0 2024-03-11T19:50:33.636 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:33.641 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 msgr2=0x7fa7b80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 0x7fa7b80a3f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4104410 tx=0x7fa7c4005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 msgr2=0x7fa7d4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7c002c6b0 tx=0x7fa7c00042f0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 shutdown_connections 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa7b80a1ac0 0x7fa7b80a3f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4104f70 0x7fa7d41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7d41045b0 0x7fa7d4199990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 --2- 172.21.15.107:0/2408040995 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa7d41033c0 0x7fa7d4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 >> 172.21.15.107:0/2408040995 conn(0x7fa7d40fe930 msgr2=0x7fa7d41078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 shutdown_connections 2024-03-11T19:50:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:33.638+0000 7fa7d9000640 1 -- 172.21.15.107:0/2408040995 wait complete. 2024-03-11T19:50:33.662 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-03-11T19:50:32.330717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597730,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":151,"num_read_kb":133,"num_write":304,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352640,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589184,"statfs":{"total":1151940427776,"available":1151579324416,"internally_reserved":0,"allocated":12079104,"data_stored":7635341,"data_compressed":283668,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2863,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":15,"num_read_kb":14,"num_write":75,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006630"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526589+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:50:02.526589+0000","last_peered":"2024-03-11T19:50:02.526589+0000","last_clean":"2024-03-11T19:50:02.526589+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:50:02.526589+0000","last_undegraded":"2024-03-11T19:50:02.526589+0000","last_fullsized":"2024-03-11T19:50:02.526589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526960+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:50:02.526960+0000","last_peered":"2024-03-11T19:50:02.526960+0000","last_clean":"2024-03-11T19:50:02.526960+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:50:02.526960+0000","last_undegraded":"2024-03-11T19:50:02.526960+0000","last_fullsized":"2024-03-11T19:50:02.526960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525943+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:50:02.525943+0000","last_peered":"2024-03-11T19:50:02.525943+0000","last_clean":"2024-03-11T19:50:02.525943+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:50:02.525943+0000","last_undegraded":"2024-03-11T19:50:02.525943+0000","last_fullsized":"2024-03-11T19:50:02.525943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526149+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:50:02.526149+0000","last_peered":"2024-03-11T19:50:02.526149+0000","last_clean":"2024-03-11T19:50:02.526149+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:50:02.526149+0000","last_undegraded":"2024-03-11T19:50:02.526149+0000","last_fullsized":"2024-03-11T19:50:02.526149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525858+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:50:02.525858+0000","last_peered":"2024-03-11T19:50:02.525858+0000","last_clean":"2024-03-11T19:50:02.525858+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:50:02.525858+0000","last_undegraded":"2024-03-11T19:50:02.525858+0000","last_fullsized":"2024-03-11T19:50:02.525858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526245+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:50:02.526245+0000","last_peered":"2024-03-11T19:50:02.526245+0000","last_clean":"2024-03-11T19:50:02.526245+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:50:02.526245+0000","last_undegraded":"2024-03-11T19:50:02.526245+0000","last_fullsized":"2024-03-11T19:50:02.526245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526904+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:50:02.526904+0000","last_peered":"2024-03-11T19:50:02.526904+0000","last_clean":"2024-03-11T19:50:02.526904+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:50:02.526904+0000","last_undegraded":"2024-03-11T19:50:02.526904+0000","last_fullsized":"2024-03-11T19:50:02.526904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.577655+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:50:01.577655+0000","last_peered":"2024-03-11T19:50:01.577655+0000","last_clean":"2024-03-11T19:50:01.577655+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:50:01.577655+0000","last_undegraded":"2024-03-11T19:50:01.577655+0000","last_fullsized":"2024-03-11T19:50:01.577655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429582+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:50:01.429582+0000","last_peered":"2024-03-11T19:50:01.429582+0000","last_clean":"2024-03-11T19:50:01.429582+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:50:01.429582+0000","last_undegraded":"2024-03-11T19:50:01.429582+0000","last_fullsized":"2024-03-11T19:50:01.429582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526775+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:50:02.526775+0000","last_peered":"2024-03-11T19:50:02.526775+0000","last_clean":"2024-03-11T19:50:02.526775+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:50:02.526775+0000","last_undegraded":"2024-03-11T19:50:02.526775+0000","last_fullsized":"2024-03-11T19:50:02.526775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527359+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:50:02.527359+0000","last_peered":"2024-03-11T19:50:02.527359+0000","last_clean":"2024-03-11T19:50:02.527359+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:50:02.527359+0000","last_undegraded":"2024-03-11T19:50:02.527359+0000","last_fullsized":"2024-03-11T19:50:02.527359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525717+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:50:02.525717+0000","last_peered":"2024-03-11T19:50:02.525717+0000","last_clean":"2024-03-11T19:50:02.525717+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:50:02.525717+0000","last_undegraded":"2024-03-11T19:50:02.525717+0000","last_fullsized":"2024-03-11T19:50:02.525717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430776+0000","last_change":"2024-03-11T19:49:55.201525+0000","last_active":"2024-03-11T19:50:01.430776+0000","last_peered":"2024-03-11T19:50:01.430776+0000","last_clean":"2024-03-11T19:50:01.430776+0000","last_became_active":"2024-03-11T19:49:55.199729+0000","last_became_peered":"2024-03-11T19:49:55.199729+0000","last_unstale":"2024-03-11T19:50:01.430776+0000","last_undegraded":"2024-03-11T19:50:01.430776+0000","last_fullsized":"2024-03-11T19:50:01.430776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:03:26.753822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525435+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:50:02.525435+0000","last_peered":"2024-03-11T19:50:02.525435+0000","last_clean":"2024-03-11T19:50:02.525435+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:50:02.525435+0000","last_undegraded":"2024-03-11T19:50:02.525435+0000","last_fullsized":"2024-03-11T19:50:02.525435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525313+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:50:02.525313+0000","last_peered":"2024-03-11T19:50:02.525313+0000","last_clean":"2024-03-11T19:50:02.525313+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:50:02.525313+0000","last_undegraded":"2024-03-11T19:50:02.525313+0000","last_fullsized":"2024-03-11T19:50:02.525313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.428918+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:50:01.428918+0000","last_peered":"2024-03-11T19:50:01.428918+0000","last_clean":"2024-03-11T19:50:01.428918+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:50:01.428918+0000","last_undegraded":"2024-03-11T19:50:01.428918+0000","last_fullsized":"2024-03-11T19:50:01.428918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526488+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:50:02.526488+0000","last_peered":"2024-03-11T19:50:02.526488+0000","last_clean":"2024-03-11T19:50:02.526488+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:50:02.526488+0000","last_undegraded":"2024-03-11T19:50:02.526488+0000","last_fullsized":"2024-03-11T19:50:02.526488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.528750+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:50:02.528750+0000","last_peered":"2024-03-11T19:50:02.528750+0000","last_clean":"2024-03-11T19:50:02.528750+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:50:02.528750+0000","last_undegraded":"2024-03-11T19:50:02.528750+0000","last_fullsized":"2024-03-11T19:50:02.528750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527178+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:50:02.527178+0000","last_peered":"2024-03-11T19:50:02.527178+0000","last_clean":"2024-03-11T19:50:02.527178+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:50:02.527178+0000","last_undegraded":"2024-03-11T19:50:02.527178+0000","last_fullsized":"2024-03-11T19:50:02.527178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526791+0000","last_change":"2024-03-11T19:50:00.466474+0000","last_active":"2024-03-11T19:50:02.526791+0000","last_peered":"2024-03-11T19:50:02.526791+0000","last_clean":"2024-03-11T19:50:02.526791+0000","last_became_active":"2024-03-11T19:50:00.466158+0000","last_became_peered":"2024-03-11T19:50:00.466158+0000","last_unstale":"2024-03-11T19:50:02.526791+0000","last_undegraded":"2024-03-11T19:50:02.526791+0000","last_fullsized":"2024-03-11T19:50:02.526791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:12:32.502464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.442652+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:50:28.442652+0000","last_peered":"2024-03-11T19:50:28.442652+0000","last_clean":"2024-03-11T19:50:28.442652+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:50:28.442652+0000","last_undegraded":"2024-03-11T19:50:28.442652+0000","last_fullsized":"2024-03-11T19:50:28.442652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578521+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:50:01.578521+0000","last_peered":"2024-03-11T19:50:01.578521+0000","last_clean":"2024-03-11T19:50:01.578521+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:50:01.578521+0000","last_undegraded":"2024-03-11T19:50:01.578521+0000","last_fullsized":"2024-03-11T19:50:01.578521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525318+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:50:02.525318+0000","last_peered":"2024-03-11T19:50:02.525318+0000","last_clean":"2024-03-11T19:50:02.525318+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:50:02.525318+0000","last_undegraded":"2024-03-11T19:50:02.525318+0000","last_fullsized":"2024-03-11T19:50:02.525318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.579057+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:50:01.579057+0000","last_peered":"2024-03-11T19:50:01.579057+0000","last_clean":"2024-03-11T19:50:01.579057+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:50:01.579057+0000","last_undegraded":"2024-03-11T19:50:01.579057+0000","last_fullsized":"2024-03-11T19:50:01.579057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526705+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:50:02.526705+0000","last_peered":"2024-03-11T19:50:02.526705+0000","last_clean":"2024-03-11T19:50:02.526705+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:50:02.526705+0000","last_undegraded":"2024-03-11T19:50:02.526705+0000","last_fullsized":"2024-03-11T19:50:02.526705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526309+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:50:02.526309+0000","last_peered":"2024-03-11T19:50:02.526309+0000","last_clean":"2024-03-11T19:50:02.526309+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:50:02.526309+0000","last_undegraded":"2024-03-11T19:50:02.526309+0000","last_fullsized":"2024-03-11T19:50:02.526309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.435657+0000","last_change":"2024-03-11T19:49:54.709796+0000","last_active":"2024-03-11T19:50:28.435657+0000","last_peered":"2024-03-11T19:50:28.435657+0000","last_clean":"2024-03-11T19:50:28.435657+0000","last_became_active":"2024-03-11T19:49:54.709073+0000","last_became_peered":"2024-03-11T19:49:54.709073+0000","last_unstale":"2024-03-11T19:50:28.435657+0000","last_undegraded":"2024-03-11T19:50:28.435657+0000","last_fullsized":"2024-03-11T19:50:28.435657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:33.720661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429349+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:50:01.429349+0000","last_peered":"2024-03-11T19:50:01.429349+0000","last_clean":"2024-03-11T19:50:01.429349+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:50:01.429349+0000","last_undegraded":"2024-03-11T19:50:01.429349+0000","last_fullsized":"2024-03-11T19:50:01.429349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525449+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:50:02.525449+0000","last_peered":"2024-03-11T19:50:02.525449+0000","last_clean":"2024-03-11T19:50:02.525449+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:50:02.525449+0000","last_undegraded":"2024-03-11T19:50:02.525449+0000","last_fullsized":"2024-03-11T19:50:02.525449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526320+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:50:02.526320+0000","last_peered":"2024-03-11T19:50:02.526320+0000","last_clean":"2024-03-11T19:50:02.526320+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:50:02.526320+0000","last_undegraded":"2024-03-11T19:50:02.526320+0000","last_fullsized":"2024-03-11T19:50:02.526320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431661+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:50:01.431661+0000","last_peered":"2024-03-11T19:50:01.431661+0000","last_clean":"2024-03-11T19:50:01.431661+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:50:01.431661+0000","last_undegraded":"2024-03-11T19:50:01.431661+0000","last_fullsized":"2024-03-11T19:50:01.431661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526978+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:50:02.526978+0000","last_peered":"2024-03-11T19:50:02.526978+0000","last_clean":"2024-03-11T19:50:02.526978+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:50:02.526978+0000","last_undegraded":"2024-03-11T19:50:02.526978+0000","last_fullsized":"2024-03-11T19:50:02.526978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.428178+0000","last_change":"2024-03-11T19:49:54.717213+0000","last_active":"2024-03-11T19:50:27.428178+0000","last_peered":"2024-03-11T19:50:27.428178+0000","last_clean":"2024-03-11T19:50:27.428178+0000","last_became_active":"2024-03-11T19:49:54.716898+0000","last_became_peered":"2024-03-11T19:49:54.716898+0000","last_unstale":"2024-03-11T19:50:27.428178+0000","last_undegraded":"2024-03-11T19:50:27.428178+0000","last_fullsized":"2024-03-11T19:50:27.428178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:33.468535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526053+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:50:02.526053+0000","last_peered":"2024-03-11T19:50:02.526053+0000","last_clean":"2024-03-11T19:50:02.526053+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:50:02.526053+0000","last_undegraded":"2024-03-11T19:50:02.526053+0000","last_fullsized":"2024-03-11T19:50:02.526053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525659+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:50:02.525659+0000","last_peered":"2024-03-11T19:50:02.525659+0000","last_clean":"2024-03-11T19:50:02.525659+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:50:02.525659+0000","last_undegraded":"2024-03-11T19:50:02.525659+0000","last_fullsized":"2024-03-11T19:50:02.525659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526198+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:50:02.526198+0000","last_peered":"2024-03-11T19:50:02.526198+0000","last_clean":"2024-03-11T19:50:02.526198+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:50:02.526198+0000","last_undegraded":"2024-03-11T19:50:02.526198+0000","last_fullsized":"2024-03-11T19:50:02.526198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431825+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:50:01.431825+0000","last_peered":"2024-03-11T19:50:01.431825+0000","last_clean":"2024-03-11T19:50:01.431825+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:50:01.431825+0000","last_undegraded":"2024-03-11T19:50:01.431825+0000","last_fullsized":"2024-03-11T19:50:01.431825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526560+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:50:02.526560+0000","last_peered":"2024-03-11T19:50:02.526560+0000","last_clean":"2024-03-11T19:50:02.526560+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:50:02.526560+0000","last_undegraded":"2024-03-11T19:50:02.526560+0000","last_fullsized":"2024-03-11T19:50:02.526560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.440392+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:50:28.440392+0000","last_peered":"2024-03-11T19:50:28.440392+0000","last_clean":"2024-03-11T19:50:28.440392+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:50:28.440392+0000","last_undegraded":"2024-03-11T19:50:28.440392+0000","last_fullsized":"2024-03-11T19:50:28.440392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.577887+0000","last_change":"2024-03-11T19:49:55.237576+0000","last_active":"2024-03-11T19:50:01.577887+0000","last_peered":"2024-03-11T19:50:01.577887+0000","last_clean":"2024-03-11T19:50:01.577887+0000","last_became_active":"2024-03-11T19:49:55.236259+0000","last_became_peered":"2024-03-11T19:49:55.236259+0000","last_unstale":"2024-03-11T19:50:01.577887+0000","last_undegraded":"2024-03-11T19:50:01.577887+0000","last_fullsized":"2024-03-11T19:50:01.577887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:05.057879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526631+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:50:02.526631+0000","last_peered":"2024-03-11T19:50:02.526631+0000","last_clean":"2024-03-11T19:50:02.526631+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:50:02.526631+0000","last_undegraded":"2024-03-11T19:50:02.526631+0000","last_fullsized":"2024-03-11T19:50:02.526631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431242+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:50:01.431242+0000","last_peered":"2024-03-11T19:50:01.431242+0000","last_clean":"2024-03-11T19:50:01.431242+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:50:01.431242+0000","last_undegraded":"2024-03-11T19:50:01.431242+0000","last_fullsized":"2024-03-11T19:50:01.431242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527456+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:50:02.527456+0000","last_peered":"2024-03-11T19:50:02.527456+0000","last_clean":"2024-03-11T19:50:02.527456+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:50:02.527456+0000","last_undegraded":"2024-03-11T19:50:02.527456+0000","last_fullsized":"2024-03-11T19:50:02.527456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578051+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:50:01.578051+0000","last_peered":"2024-03-11T19:50:01.578051+0000","last_clean":"2024-03-11T19:50:01.578051+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:50:01.578051+0000","last_undegraded":"2024-03-11T19:50:01.578051+0000","last_fullsized":"2024-03-11T19:50:01.578051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.427778+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:50:27.427778+0000","last_peered":"2024-03-11T19:50:27.427778+0000","last_clean":"2024-03-11T19:50:27.427778+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:50:27.427778+0000","last_undegraded":"2024-03-11T19:50:27.427778+0000","last_fullsized":"2024-03-11T19:50:27.427778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429560+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:50:01.429560+0000","last_peered":"2024-03-11T19:50:01.429560+0000","last_clean":"2024-03-11T19:50:01.429560+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:50:01.429560+0000","last_undegraded":"2024-03-11T19:50:01.429560+0000","last_fullsized":"2024-03-11T19:50:01.429560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527065+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:50:02.527065+0000","last_peered":"2024-03-11T19:50:02.527065+0000","last_clean":"2024-03-11T19:50:02.527065+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:50:02.527065+0000","last_undegraded":"2024-03-11T19:50:02.527065+0000","last_fullsized":"2024-03-11T19:50:02.527065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.433576+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:50:27.433576+0000","last_peered":"2024-03-11T19:50:27.433576+0000","last_clean":"2024-03-11T19:50:27.433576+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:50:27.433576+0000","last_undegraded":"2024-03-11T19:50:27.433576+0000","last_fullsized":"2024-03-11T19:50:27.433576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526646+0000","last_change":"2024-03-11T19:49:55.221391+0000","last_active":"2024-03-11T19:50:02.526646+0000","last_peered":"2024-03-11T19:50:02.526646+0000","last_clean":"2024-03-11T19:50:02.526646+0000","last_became_active":"2024-03-11T19:49:55.212401+0000","last_became_peered":"2024-03-11T19:49:55.212401+0000","last_unstale":"2024-03-11T19:50:02.526646+0000","last_undegraded":"2024-03-11T19:50:02.526646+0000","last_fullsized":"2024-03-11T19:50:02.526646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:29:20.005530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525780+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:50:02.525780+0000","last_peered":"2024-03-11T19:50:02.525780+0000","last_clean":"2024-03-11T19:50:02.525780+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:50:02.525780+0000","last_undegraded":"2024-03-11T19:50:02.525780+0000","last_fullsized":"2024-03-11T19:50:02.525780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431859+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:50:01.431859+0000","last_peered":"2024-03-11T19:50:01.431859+0000","last_clean":"2024-03-11T19:50:01.431859+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:50:01.431859+0000","last_undegraded":"2024-03-11T19:50:01.431859+0000","last_fullsized":"2024-03-11T19:50:01.431859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429539+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:50:01.429539+0000","last_peered":"2024-03-11T19:50:01.429539+0000","last_clean":"2024-03-11T19:50:01.429539+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:50:01.429539+0000","last_undegraded":"2024-03-11T19:50:01.429539+0000","last_fullsized":"2024-03-11T19:50:01.429539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527757+0000","last_change":"2024-03-11T19:50:00.467047+0000","last_active":"2024-03-11T19:50:02.527757+0000","last_peered":"2024-03-11T19:50:02.527757+0000","last_clean":"2024-03-11T19:50:02.527757+0000","last_became_active":"2024-03-11T19:50:00.466637+0000","last_became_peered":"2024-03-11T19:50:00.466637+0000","last_unstale":"2024-03-11T19:50:02.527757+0000","last_undegraded":"2024-03-11T19:50:02.527757+0000","last_fullsized":"2024-03-11T19:50:02.527757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:42:00.183583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.426019+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:50:27.426019+0000","last_peered":"2024-03-11T19:50:27.426019+0000","last_clean":"2024-03-11T19:50:27.426019+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:50:27.426019+0000","last_undegraded":"2024-03-11T19:50:27.426019+0000","last_fullsized":"2024-03-11T19:50:27.426019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526855+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:50:02.526855+0000","last_peered":"2024-03-11T19:50:02.526855+0000","last_clean":"2024-03-11T19:50:02.526855+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:50:02.526855+0000","last_undegraded":"2024-03-11T19:50:02.526855+0000","last_fullsized":"2024-03-11T19:50:02.526855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.432219+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:50:01.432219+0000","last_peered":"2024-03-11T19:50:01.432219+0000","last_clean":"2024-03-11T19:50:01.432219+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:50:01.432219+0000","last_undegraded":"2024-03-11T19:50:01.432219+0000","last_fullsized":"2024-03-11T19:50:01.432219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.436620+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:50:28.436620+0000","last_peered":"2024-03-11T19:50:28.436620+0000","last_clean":"2024-03-11T19:50:28.436620+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:50:28.436620+0000","last_undegraded":"2024-03-11T19:50:28.436620+0000","last_fullsized":"2024-03-11T19:50:28.436620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430613+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:50:01.430613+0000","last_peered":"2024-03-11T19:50:01.430613+0000","last_clean":"2024-03-11T19:50:01.430613+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:50:01.430613+0000","last_undegraded":"2024-03-11T19:50:01.430613+0000","last_fullsized":"2024-03-11T19:50:01.430613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525716+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:50:02.525716+0000","last_peered":"2024-03-11T19:50:02.525716+0000","last_clean":"2024-03-11T19:50:02.525716+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:50:02.525716+0000","last_undegraded":"2024-03-11T19:50:02.525716+0000","last_fullsized":"2024-03-11T19:50:02.525716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.429530+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:50:27.429530+0000","last_peered":"2024-03-11T19:50:27.429530+0000","last_clean":"2024-03-11T19:50:27.429530+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:50:27.429530+0000","last_undegraded":"2024-03-11T19:50:27.429530+0000","last_fullsized":"2024-03-11T19:50:27.429530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527048+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:50:02.527048+0000","last_peered":"2024-03-11T19:50:02.527048+0000","last_clean":"2024-03-11T19:50:02.527048+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:50:02.527048+0000","last_undegraded":"2024-03-11T19:50:02.527048+0000","last_fullsized":"2024-03-11T19:50:02.527048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527952+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:50:02.527952+0000","last_peered":"2024-03-11T19:50:02.527952+0000","last_clean":"2024-03-11T19:50:02.527952+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:50:02.527952+0000","last_undegraded":"2024-03-11T19:50:02.527952+0000","last_fullsized":"2024-03-11T19:50:02.527952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:29.447885+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:50:29.447885+0000","last_peered":"2024-03-11T19:50:29.447885+0000","last_clean":"2024-03-11T19:50:29.447885+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:50:29.447885+0000","last_undegraded":"2024-03-11T19:50:29.447885+0000","last_fullsized":"2024-03-11T19:50:29.447885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526383+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:50:02.526383+0000","last_peered":"2024-03-11T19:50:02.526383+0000","last_clean":"2024-03-11T19:50:02.526383+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:50:02.526383+0000","last_undegraded":"2024-03-11T19:50:02.526383+0000","last_fullsized":"2024-03-11T19:50:02.526383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431739+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:50:01.431739+0000","last_peered":"2024-03-11T19:50:01.431739+0000","last_clean":"2024-03-11T19:50:01.431739+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:50:01.431739+0000","last_undegraded":"2024-03-11T19:50:01.431739+0000","last_fullsized":"2024-03-11T19:50:01.431739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.439341+0000","last_change":"2024-03-11T19:49:54.708749+0000","last_active":"2024-03-11T19:50:28.439341+0000","last_peered":"2024-03-11T19:50:28.439341+0000","last_clean":"2024-03-11T19:50:28.439341+0000","last_became_active":"2024-03-11T19:49:54.708246+0000","last_became_peered":"2024-03-11T19:49:54.708246+0000","last_unstale":"2024-03-11T19:50:28.439341+0000","last_undegraded":"2024-03-11T19:50:28.439341+0000","last_fullsized":"2024-03-11T19:50:28.439341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:50.434110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431402+0000","last_change":"2024-03-11T19:49:55.214263+0000","last_active":"2024-03-11T19:50:01.431402+0000","last_peered":"2024-03-11T19:50:01.431402+0000","last_clean":"2024-03-11T19:50:01.431402+0000","last_became_active":"2024-03-11T19:49:55.213940+0000","last_became_peered":"2024-03-11T19:49:55.213940+0000","last_unstale":"2024-03-11T19:50:01.431402+0000","last_undegraded":"2024-03-11T19:50:01.431402+0000","last_fullsized":"2024-03-11T19:50:01.431402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:28.733906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431572+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:50:01.431572+0000","last_peered":"2024-03-11T19:50:01.431572+0000","last_clean":"2024-03-11T19:50:01.431572+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:50:01.431572+0000","last_undegraded":"2024-03-11T19:50:01.431572+0000","last_fullsized":"2024-03-11T19:50:01.431572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526735+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:50:02.526735+0000","last_peered":"2024-03-11T19:50:02.526735+0000","last_clean":"2024-03-11T19:50:02.526735+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:50:02.526735+0000","last_undegraded":"2024-03-11T19:50:02.526735+0000","last_fullsized":"2024-03-11T19:50:02.526735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526213+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:50:02.526213+0000","last_peered":"2024-03-11T19:50:02.526213+0000","last_clean":"2024-03-11T19:50:02.526213+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:50:02.526213+0000","last_undegraded":"2024-03-11T19:50:02.526213+0000","last_fullsized":"2024-03-11T19:50:02.526213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525906+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:50:02.525906+0000","last_peered":"2024-03-11T19:50:02.525906+0000","last_clean":"2024-03-11T19:50:02.525906+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:50:02.525906+0000","last_undegraded":"2024-03-11T19:50:02.525906+0000","last_fullsized":"2024-03-11T19:50:02.525906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.528380+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:50:02.528380+0000","last_peered":"2024-03-11T19:50:02.528380+0000","last_clean":"2024-03-11T19:50:02.528380+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:50:02.528380+0000","last_undegraded":"2024-03-11T19:50:02.528380+0000","last_fullsized":"2024-03-11T19:50:02.528380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526636+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:50:02.526636+0000","last_peered":"2024-03-11T19:50:02.526636+0000","last_clean":"2024-03-11T19:50:02.526636+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:50:02.526636+0000","last_undegraded":"2024-03-11T19:50:02.526636+0000","last_fullsized":"2024-03-11T19:50:02.526636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430960+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:50:01.430960+0000","last_peered":"2024-03-11T19:50:01.430960+0000","last_clean":"2024-03-11T19:50:01.430960+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:50:01.430960+0000","last_undegraded":"2024-03-11T19:50:01.430960+0000","last_fullsized":"2024-03-11T19:50:01.430960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527254+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:50:02.527254+0000","last_peered":"2024-03-11T19:50:02.527254+0000","last_clean":"2024-03-11T19:50:02.527254+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:50:02.527254+0000","last_undegraded":"2024-03-11T19:50:02.527254+0000","last_fullsized":"2024-03-11T19:50:02.527254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526366+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:50:02.526366+0000","last_peered":"2024-03-11T19:50:02.526366+0000","last_clean":"2024-03-11T19:50:02.526366+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:50:02.526366+0000","last_undegraded":"2024-03-11T19:50:02.526366+0000","last_fullsized":"2024-03-11T19:50:02.526366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431624+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:50:01.431624+0000","last_peered":"2024-03-11T19:50:01.431624+0000","last_clean":"2024-03-11T19:50:01.431624+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:50:01.431624+0000","last_undegraded":"2024-03-11T19:50:01.431624+0000","last_fullsized":"2024-03-11T19:50:01.431624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431371+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:50:01.431371+0000","last_peered":"2024-03-11T19:50:01.431371+0000","last_clean":"2024-03-11T19:50:01.431371+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:50:01.431371+0000","last_undegraded":"2024-03-11T19:50:01.431371+0000","last_fullsized":"2024-03-11T19:50:01.431371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.438920+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:50:28.438920+0000","last_peered":"2024-03-11T19:50:28.438920+0000","last_clean":"2024-03-11T19:50:28.438920+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:50:28.438920+0000","last_undegraded":"2024-03-11T19:50:28.438920+0000","last_fullsized":"2024-03-11T19:50:28.438920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527139+0000","last_change":"2024-03-11T19:50:00.464425+0000","last_active":"2024-03-11T19:50:02.527139+0000","last_peered":"2024-03-11T19:50:02.527139+0000","last_clean":"2024-03-11T19:50:02.527139+0000","last_became_active":"2024-03-11T19:50:00.463990+0000","last_became_peered":"2024-03-11T19:50:00.463990+0000","last_unstale":"2024-03-11T19:50:02.527139+0000","last_undegraded":"2024-03-11T19:50:02.527139+0000","last_fullsized":"2024-03-11T19:50:02.527139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:28:29.975190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578192+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:50:01.578192+0000","last_peered":"2024-03-11T19:50:01.578192+0000","last_clean":"2024-03-11T19:50:01.578192+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:50:01.578192+0000","last_undegraded":"2024-03-11T19:50:01.578192+0000","last_fullsized":"2024-03-11T19:50:01.578192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:29.445439+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:50:29.445439+0000","last_peered":"2024-03-11T19:50:29.445439+0000","last_clean":"2024-03-11T19:50:29.445439+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:50:29.445439+0000","last_undegraded":"2024-03-11T19:50:29.445439+0000","last_fullsized":"2024-03-11T19:50:29.445439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526176+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:50:02.526176+0000","last_peered":"2024-03-11T19:50:02.526176+0000","last_clean":"2024-03-11T19:50:02.526176+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:50:02.526176+0000","last_undegraded":"2024-03-11T19:50:02.526176+0000","last_fullsized":"2024-03-11T19:50:02.526176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.432019+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:50:01.432019+0000","last_peered":"2024-03-11T19:50:01.432019+0000","last_clean":"2024-03-11T19:50:01.432019+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:50:01.432019+0000","last_undegraded":"2024-03-11T19:50:01.432019+0000","last_fullsized":"2024-03-11T19:50:01.432019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430005+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:50:01.430005+0000","last_peered":"2024-03-11T19:50:01.430005+0000","last_clean":"2024-03-11T19:50:01.430005+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:50:01.430005+0000","last_undegraded":"2024-03-11T19:50:01.430005+0000","last_fullsized":"2024-03-11T19:50:01.430005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.435599+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:50:28.435599+0000","last_peered":"2024-03-11T19:50:28.435599+0000","last_clean":"2024-03-11T19:50:28.435599+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:50:28.435599+0000","last_undegraded":"2024-03-11T19:50:28.435599+0000","last_fullsized":"2024-03-11T19:50:28.435599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525648+0000","last_change":"2024-03-11T19:50:00.468421+0000","last_active":"2024-03-11T19:50:02.525648+0000","last_peered":"2024-03-11T19:50:02.525648+0000","last_clean":"2024-03-11T19:50:02.525648+0000","last_became_active":"2024-03-11T19:50:00.467973+0000","last_became_peered":"2024-03-11T19:50:00.467973+0000","last_unstale":"2024-03-11T19:50:02.525648+0000","last_undegraded":"2024-03-11T19:50:02.525648+0000","last_fullsized":"2024-03-11T19:50:02.525648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:50:56.817000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.431634+0000","last_change":"2024-03-11T19:49:54.720835+0000","last_active":"2024-03-11T19:50:27.431634+0000","last_peered":"2024-03-11T19:50:27.431634+0000","last_clean":"2024-03-11T19:50:27.431634+0000","last_became_active":"2024-03-11T19:49:54.720436+0000","last_became_peered":"2024-03-11T19:49:54.720436+0000","last_unstale":"2024-03-11T19:50:27.431634+0000","last_undegraded":"2024-03-11T19:50:27.431634+0000","last_fullsized":"2024-03-11T19:50:27.431634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:20:08.706822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431566+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:50:01.431566+0000","last_peered":"2024-03-11T19:50:01.431566+0000","last_clean":"2024-03-11T19:50:01.431566+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:50:01.431566+0000","last_undegraded":"2024-03-11T19:50:01.431566+0000","last_fullsized":"2024-03-11T19:50:01.431566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578063+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:50:01.578063+0000","last_peered":"2024-03-11T19:50:01.578063+0000","last_clean":"2024-03-11T19:50:01.578063+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:50:01.578063+0000","last_undegraded":"2024-03-11T19:50:01.578063+0000","last_fullsized":"2024-03-11T19:50:01.578063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.441762+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:50:28.441762+0000","last_peered":"2024-03-11T19:50:28.441762+0000","last_clean":"2024-03-11T19:50:28.441762+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:50:28.441762+0000","last_undegraded":"2024-03-11T19:50:28.441762+0000","last_fullsized":"2024-03-11T19:50:28.441762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578935+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:50:01.578935+0000","last_peered":"2024-03-11T19:50:01.578935+0000","last_clean":"2024-03-11T19:50:01.578935+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:50:01.578935+0000","last_undegraded":"2024-03-11T19:50:01.578935+0000","last_fullsized":"2024-03-11T19:50:01.578935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526534+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:50:02.526534+0000","last_peered":"2024-03-11T19:50:02.526534+0000","last_clean":"2024-03-11T19:50:02.526534+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:50:02.526534+0000","last_undegraded":"2024-03-11T19:50:02.526534+0000","last_fullsized":"2024-03-11T19:50:02.526534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.425913+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:50:27.425913+0000","last_peered":"2024-03-11T19:50:27.425913+0000","last_clean":"2024-03-11T19:50:27.425913+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:50:27.425913+0000","last_undegraded":"2024-03-11T19:50:27.425913+0000","last_fullsized":"2024-03-11T19:50:27.425913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527349+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:50:02.527349+0000","last_peered":"2024-03-11T19:50:02.527349+0000","last_clean":"2024-03-11T19:50:02.527349+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:50:02.527349+0000","last_undegraded":"2024-03-11T19:50:02.527349+0000","last_fullsized":"2024-03-11T19:50:02.527349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525774+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:50:02.525774+0000","last_peered":"2024-03-11T19:50:02.525774+0000","last_clean":"2024-03-11T19:50:02.525774+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:50:02.525774+0000","last_undegraded":"2024-03-11T19:50:02.525774+0000","last_fullsized":"2024-03-11T19:50:02.525774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.433707+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:50:27.433707+0000","last_peered":"2024-03-11T19:50:27.433707+0000","last_clean":"2024-03-11T19:50:27.433707+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:50:27.433707+0000","last_undegraded":"2024-03-11T19:50:27.433707+0000","last_fullsized":"2024-03-11T19:50:27.433707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525586+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:50:02.525586+0000","last_peered":"2024-03-11T19:50:02.525586+0000","last_clean":"2024-03-11T19:50:02.525586+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:50:02.525586+0000","last_undegraded":"2024-03-11T19:50:02.525586+0000","last_fullsized":"2024-03-11T19:50:02.525586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.529111+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:50:02.529111+0000","last_peered":"2024-03-11T19:50:02.529111+0000","last_clean":"2024-03-11T19:50:02.529111+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:50:02.529111+0000","last_undegraded":"2024-03-11T19:50:02.529111+0000","last_fullsized":"2024-03-11T19:50:02.529111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578122+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:50:01.578122+0000","last_peered":"2024-03-11T19:50:01.578122+0000","last_clean":"2024-03-11T19:50:01.578122+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:50:01.578122+0000","last_undegraded":"2024-03-11T19:50:01.578122+0000","last_fullsized":"2024-03-11T19:50:01.578122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525225+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:50:02.525225+0000","last_peered":"2024-03-11T19:50:02.525225+0000","last_clean":"2024-03-11T19:50:02.525225+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:50:02.525225+0000","last_undegraded":"2024-03-11T19:50:02.525225+0000","last_fullsized":"2024-03-11T19:50:02.525225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526509+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:50:02.526509+0000","last_peered":"2024-03-11T19:50:02.526509+0000","last_clean":"2024-03-11T19:50:02.526509+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:50:02.526509+0000","last_undegraded":"2024-03-11T19:50:02.526509+0000","last_fullsized":"2024-03-11T19:50:02.526509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525522+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:50:02.525522+0000","last_peered":"2024-03-11T19:50:02.525522+0000","last_clean":"2024-03-11T19:50:02.525522+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:50:02.525522+0000","last_undegraded":"2024-03-11T19:50:02.525522+0000","last_fullsized":"2024-03-11T19:50:02.525522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578297+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:50:01.578297+0000","last_peered":"2024-03-11T19:50:01.578297+0000","last_clean":"2024-03-11T19:50:01.578297+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:50:01.578297+0000","last_undegraded":"2024-03-11T19:50:01.578297+0000","last_fullsized":"2024-03-11T19:50:01.578297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525710+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:50:02.525710+0000","last_peered":"2024-03-11T19:50:02.525710+0000","last_clean":"2024-03-11T19:50:02.525710+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:50:02.525710+0000","last_undegraded":"2024-03-11T19:50:02.525710+0000","last_fullsized":"2024-03-11T19:50:02.525710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:27.428801+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:50:27.428801+0000","last_peered":"2024-03-11T19:50:27.428801+0000","last_clean":"2024-03-11T19:50:27.428801+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:50:27.428801+0000","last_undegraded":"2024-03-11T19:50:27.428801+0000","last_fullsized":"2024-03-11T19:50:27.428801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527075+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:50:02.527075+0000","last_peered":"2024-03-11T19:50:02.527075+0000","last_clean":"2024-03-11T19:50:02.527075+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:50:02.527075+0000","last_undegraded":"2024-03-11T19:50:02.527075+0000","last_fullsized":"2024-03-11T19:50:02.527075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429162+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:50:01.429162+0000","last_peered":"2024-03-11T19:50:01.429162+0000","last_clean":"2024-03-11T19:50:01.429162+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:50:01.429162+0000","last_undegraded":"2024-03-11T19:50:01.429162+0000","last_fullsized":"2024-03-11T19:50:01.429162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:28.438714+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:50:28.438714+0000","last_peered":"2024-03-11T19:50:28.438714+0000","last_clean":"2024-03-11T19:50:28.438714+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:50:28.438714+0000","last_undegraded":"2024-03-11T19:50:28.438714+0000","last_fullsized":"2024-03-11T19:50:28.438714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527391+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:50:02.527391+0000","last_peered":"2024-03-11T19:50:02.527391+0000","last_clean":"2024-03-11T19:50:02.527391+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:50:02.527391+0000","last_undegraded":"2024-03-11T19:50:02.527391+0000","last_fullsized":"2024-03-11T19:50:02.527391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525736+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:50:02.525736+0000","last_peered":"2024-03-11T19:50:02.525736+0000","last_clean":"2024-03-11T19:50:02.525736+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:50:02.525736+0000","last_undegraded":"2024-03-11T19:50:02.525736+0000","last_fullsized":"2024-03-11T19:50:02.525736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431951+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:50:01.431951+0000","last_peered":"2024-03-11T19:50:01.431951+0000","last_clean":"2024-03-11T19:50:01.431951+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:50:01.431951+0000","last_undegraded":"2024-03-11T19:50:01.431951+0000","last_fullsized":"2024-03-11T19:50:01.431951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527094+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:50:02.527094+0000","last_peered":"2024-03-11T19:50:02.527094+0000","last_clean":"2024-03-11T19:50:02.527094+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:50:02.527094+0000","last_undegraded":"2024-03-11T19:50:02.527094+0000","last_fullsized":"2024-03-11T19:50:02.527094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431228+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:50:01.431228+0000","last_peered":"2024-03-11T19:50:01.431228+0000","last_clean":"2024-03-11T19:50:01.431228+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:50:01.431228+0000","last_undegraded":"2024-03-11T19:50:01.431228+0000","last_fullsized":"2024-03-11T19:50:01.431228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431436+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:50:01.431436+0000","last_peered":"2024-03-11T19:50:01.431436+0000","last_clean":"2024-03-11T19:50:01.431436+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:50:01.431436+0000","last_undegraded":"2024-03-11T19:50:01.431436+0000","last_fullsized":"2024-03-11T19:50:01.431436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.527535+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:50:02.527535+0000","last_peered":"2024-03-11T19:50:02.527535+0000","last_clean":"2024-03-11T19:50:02.527535+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:50:02.527535+0000","last_undegraded":"2024-03-11T19:50:02.527535+0000","last_fullsized":"2024-03-11T19:50:02.527535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525321+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:50:02.525321+0000","last_peered":"2024-03-11T19:50:02.525321+0000","last_clean":"2024-03-11T19:50:02.525321+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:50:02.525321+0000","last_undegraded":"2024-03-11T19:50:02.525321+0000","last_fullsized":"2024-03-11T19:50:02.525321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526673+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:50:02.526673+0000","last_peered":"2024-03-11T19:50:02.526673+0000","last_clean":"2024-03-11T19:50:02.526673+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:50:02.526673+0000","last_undegraded":"2024-03-11T19:50:02.526673+0000","last_fullsized":"2024-03-11T19:50:02.526673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525343+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:50:02.525343+0000","last_peered":"2024-03-11T19:50:02.525343+0000","last_clean":"2024-03-11T19:50:02.525343+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:50:02.525343+0000","last_undegraded":"2024-03-11T19:50:02.525343+0000","last_fullsized":"2024-03-11T19:50:02.525343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526053+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:50:02.526053+0000","last_peered":"2024-03-11T19:50:02.526053+0000","last_clean":"2024-03-11T19:50:02.526053+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:50:02.526053+0000","last_undegraded":"2024-03-11T19:50:02.526053+0000","last_fullsized":"2024-03-11T19:50:02.526053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431576+0000","last_change":"2024-03-11T19:49:55.218937+0000","last_active":"2024-03-11T19:50:01.431576+0000","last_peered":"2024-03-11T19:50:01.431576+0000","last_clean":"2024-03-11T19:50:01.431576+0000","last_became_active":"2024-03-11T19:49:55.218621+0000","last_became_peered":"2024-03-11T19:49:55.218621+0000","last_unstale":"2024-03-11T19:50:01.431576+0000","last_undegraded":"2024-03-11T19:50:01.431576+0000","last_fullsized":"2024-03-11T19:50:01.431576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:56.928038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526257+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:50:02.526257+0000","last_peered":"2024-03-11T19:50:02.526257+0000","last_clean":"2024-03-11T19:50:02.526257+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:50:02.526257+0000","last_undegraded":"2024-03-11T19:50:02.526257+0000","last_fullsized":"2024-03-11T19:50:02.526257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.526160+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:50:02.526160+0000","last_peered":"2024-03-11T19:50:02.526160+0000","last_clean":"2024-03-11T19:50:02.526160+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:50:02.526160+0000","last_undegraded":"2024-03-11T19:50:02.526160+0000","last_fullsized":"2024-03-11T19:50:02.526160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.579077+0000","last_change":"2024-03-11T19:49:55.232791+0000","last_active":"2024-03-11T19:50:01.579077+0000","last_peered":"2024-03-11T19:50:01.579077+0000","last_clean":"2024-03-11T19:50:01.579077+0000","last_became_active":"2024-03-11T19:49:55.229966+0000","last_became_peered":"2024-03-11T19:49:55.229966+0000","last_unstale":"2024-03-11T19:50:01.579077+0000","last_undegraded":"2024-03-11T19:50:01.579077+0000","last_fullsized":"2024-03-11T19:50:01.579077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:17.049558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.431305+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:50:01.431305+0000","last_peered":"2024-03-11T19:50:01.431305+0000","last_clean":"2024-03-11T19:50:01.431305+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:50:01.431305+0000","last_undegraded":"2024-03-11T19:50:01.431305+0000","last_fullsized":"2024-03-11T19:50:01.431305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.578308+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:50:01.578308+0000","last_peered":"2024-03-11T19:50:01.578308+0000","last_clean":"2024-03-11T19:50:01.578308+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:50:01.578308+0000","last_undegraded":"2024-03-11T19:50:01.578308+0000","last_fullsized":"2024-03-11T19:50:01.578308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.429233+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:50:01.429233+0000","last_peered":"2024-03-11T19:50:01.429233+0000","last_clean":"2024-03-11T19:50:01.429233+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:50:01.429233+0000","last_undegraded":"2024-03-11T19:50:01.429233+0000","last_fullsized":"2024-03-11T19:50:01.429233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:01.430857+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:50:01.430857+0000","last_peered":"2024-03-11T19:50:01.430857+0000","last_clean":"2024-03-11T19:50:01.430857+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:50:01.430857+0000","last_undegraded":"2024-03-11T19:50:01.430857+0000","last_fullsized":"2024-03-11T19:50:01.430857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-11T19:50:02.525859+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:50:02.525859+0000","last_peered":"2024-03-11T19:50:02.525859+0000","last_clean":"2024-03-11T19:50:02.525859+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:50:02.525859+0000","last_undegraded":"2024-03-11T19:50:02.525859+0000","last_fullsized":"2024-03-11T19:50:02.525859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7362,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":15,"num_read_kb":14,"num_write":123,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":65374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":485394,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1079798,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486212,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":486132,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1077554,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":485484,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":508535,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":486872,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490586,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":485498,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486038,"data_compressed":14685,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1077238,"data_compressed":50501,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":12183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:50:33.667 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-11T19:50:33.668 INFO:tasks.cephadm:Setting up client nodes... 2024-03-11T19:50:33.668 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-11T19:50:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:33 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1562714380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:33 smithi122 ceph-mon[36397]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:34.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:33 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1562714380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:34.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:33 smithi145 ceph-mon[36730]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:33 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1562714380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:33 smithi107 ceph-mon[31712]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:34.139 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:34.815 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:34 smithi107 ceph-mon[31712]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:34.815 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:34 smithi107 ceph-mon[31712]: pgmap v340: 129 pgs: 129 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:50:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:34 smithi122 ceph-mon[36397]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:34 smithi122 ceph-mon[36397]: pgmap v340: 129 pgs: 129 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:50:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:34 smithi145 ceph-mon[36730]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:50:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:34 smithi145 ceph-mon[36730]: pgmap v340: 129 pgs: 129 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 -- 172.21.15.107:0/3603423564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8130105080 msgr2=0x7f8130107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 --2- 172.21.15.107:0/3603423564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8130105080 0x7f8130107470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f811c0098f0 tx=0x7f811c009c00 comp rx=0 tx=0).stop 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 -- 172.21.15.107:0/3603423564 shutdown_connections 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 --2- 172.21.15.107:0/3603423564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81301079b0 0x7f8130109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 --2- 172.21.15.107:0/3603423564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 --2- 172.21.15.107:0/3603423564 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8130105080 0x7f8130107470 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 -- 172.21.15.107:0/3603423564 >> 172.21.15.107:0/3603423564 conn(0x7f81300fc180 msgr2=0x7f81300fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.559+0000 7f8138600700 1 -- 172.21.15.107:0/3603423564 shutdown_connections 2024-03-11T19:50:36.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.560+0000 7f8138600700 1 -- 172.21.15.107:0/3603423564 wait complete. 2024-03-11T19:50:36.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.560+0000 7f8138600700 1 Processor -- start 2024-03-11T19:50:36.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 -- start start 2024-03-11T19:50:36.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81301079b0 0x7f81301a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301a1990 con 0x7f8130105080 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301a1ad0 con 0x7f8130102750 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8138600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301a1c60 con 0x7f81301079b0 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8135200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8135200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37984/0 (socket says 172.21.15.107:37984) 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8135200700 1 -- 172.21.15.107:0/4277164197 learned_addr learned my addr 172.21.15.107:0/4277164197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.561+0000 7f8135c00700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8136600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81301079b0 0x7f81301a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 -- 172.21.15.107:0/4277164197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81301079b0 msgr2=0x7f81301a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81301079b0 0x7f81301a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 -- 172.21.15.107:0/4277164197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 msgr2=0x7f8130199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 -- 172.21.15.107:0/4277164197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128009d00 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135c00700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.562+0000 7f8135200700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f811c009810 tx=0x7f811c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.563+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c01c070 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.563+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c009520 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.563+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f811c005650 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.563+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c003d70 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.563+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81301a21c0 con 0x7f8130105080 2024-03-11T19:50:36.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.564+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8130109580 con 0x7f8130105080 2024-03-11T19:50:36.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.566+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f811c00b560 con 0x7f8130105080 2024-03-11T19:50:36.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.568+0000 7f812e000700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 0x7f81180a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.568+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f811c0ce4b0 con 0x7f8130105080 2024-03-11T19:50:36.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.568+0000 7f8135c00700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 0x7f81180a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.569+0000 7f8135c00700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 0x7f81180a40c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8124011490 tx=0x7f8124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.572+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f811c097350 con 0x7f8130105080 2024-03-11T19:50:36.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.876+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f813010d6c0 con 0x7f8130105080 2024-03-11T19:50:36.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.883+0000 7f812e000700 1 -- 172.21.15.107:0/4277164197 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f811c09aa10 con 0x7f8130105080 2024-03-11T19:50:36.888 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-03-11T19:50:36.888 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCMYO9lxH1kNBAAVuBhL/Wu7pIfbUvIIfW5Gw== 2024-03-11T19:50:36.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 msgr2=0x7f81180a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 0x7f81180a40c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8124011490 tx=0x7f8124009260 comp rx=0 tx=0).stop 2024-03-11T19:50:36.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 msgr2=0x7f813019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f811c009810 tx=0x7f811c005250 comp rx=0 tx=0).stop 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 shutdown_connections 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f81180a1c00 0x7f81180a40c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81301079b0 0x7f81301a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8130102750 0x7f8130199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 --2- 172.21.15.107:0/4277164197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130105080 0x7f813019a340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 >> 172.21.15.107:0/4277164197 conn(0x7f81300fc180 msgr2=0x7f8130101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.886+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 shutdown_connections 2024-03-11T19:50:36.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:36.887+0000 7f8138600700 1 -- 172.21.15.107:0/4277164197 wait complete. 2024-03-11T19:50:37.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:37 smithi107 ceph-mon[31712]: pgmap v341: 129 pgs: 129 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-03-11T19:50:37.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:37 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:50:37.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:37 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:50:37.740 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:50:37.740 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-11T19:50:37.740 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-11T19:50:37.741 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:37 smithi145 ceph-mon[36730]: pgmap v341: 129 pgs: 129 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-03-11T19:50:37.741 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:37 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:50:37.741 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:37 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:50:37.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:37 smithi122 ceph-mon[36397]: pgmap v341: 129 pgs: 129 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-03-11T19:50:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:37 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:50:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:37 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4277164197' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:50:37.879 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-11T19:50:37.879 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-11T19:50:37.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-11T19:50:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:39 smithi122 ceph-mon[36397]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-11T19:50:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:39 smithi145 ceph-mon[36730]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-11T19:50:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:39 smithi107 ceph-mon[31712]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 -- 172.21.15.107:0/2298435544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 msgr2=0x7efca8115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 --2- 172.21.15.107:0/2298435544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca8115060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efca000b0a0 tx=0x7efca000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 -- 172.21.15.107:0/2298435544 shutdown_connections 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 --2- 172.21.15.107:0/2298435544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca8115060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 --2- 172.21.15.107:0/2298435544 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 --2- 172.21.15.107:0/2298435544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca810cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 -- 172.21.15.107:0/2298435544 >> 172.21.15.107:0/2298435544 conn(0x7efca8077910 msgr2=0x7efca8079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:40.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 -- 172.21.15.107:0/2298435544 shutdown_connections 2024-03-11T19:50:40.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.673+0000 7efcacc00700 1 -- 172.21.15.107:0/2298435544 wait complete. 2024-03-11T19:50:40.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.674+0000 7efcacc00700 1 Processor -- start 2024-03-11T19:50:40.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.674+0000 7efcacc00700 1 -- start start 2024-03-11T19:50:40.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca81a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca819ec50 con 0x7efca810d490 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca819ed90 con 0x7efca8108db0 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efcacc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca819ef50 con 0x7efca8108400 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca7400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38010/0 (socket says 172.21.15.107:38010) 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca7400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38806/0 (socket says 172.21.15.107:38806) 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca7e00700 1 -- 172.21.15.107:0/504668400 learned_addr learned my addr 172.21.15.107:0/504668400 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.675+0000 7efca6a00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca81a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 -- 172.21.15.107:0/504668400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 msgr2=0x7efca81a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca81a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 -- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 msgr2=0x7efca819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 -- 172.21.15.107:0/504668400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0009d00 con 0x7efca8108400 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7e00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca6a00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca81a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efca7400700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9f0 tx=0x7efc9000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000c9c0 con 0x7efca8108400 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.676+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca819f230 con 0x7efca8108400 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.677+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca81ab480 con 0x7efca8108400 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.677+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc90010430 con 0x7efca8108400 2024-03-11T19:50:40.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.677+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90018680 con 0x7efca8108400 2024-03-11T19:50:40.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.678+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc84005330 con 0x7efca8108400 2024-03-11T19:50:40.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.680+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7efc900105a0 con 0x7efca8108400 2024-03-11T19:50:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.681+0000 7efc9fe00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 0x7efc8c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.681+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efc90014070 con 0x7efca8108400 2024-03-11T19:50:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.681+0000 7efca6a00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 0x7efc8c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.682+0000 7efca6a00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 0x7efc8c0a4090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efca8107560 tx=0x7efc98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.683+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc90052d20 con 0x7efca8108400 2024-03-11T19:50:40.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.962+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efc840058b0 con 0x7efca8108400 2024-03-11T19:50:40.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.968+0000 7efc9fe00700 1 -- 172.21.15.107:0/504668400 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260671 (secure 0 0 0) 0x7efc90065a50 con 0x7efca8108400 2024-03-11T19:50:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 msgr2=0x7efc8c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 0x7efc8c0a4090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efca8107560 tx=0x7efc98009260 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 msgr2=0x7efca81a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9f0 tx=0x7efc9000ed00 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 shutdown_connections 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efc8c0a1bd0 0x7efc8c0a4090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efca810d490 0x7efca819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efca8108400 0x7efca81a43e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 --2- 172.21.15.107:0/504668400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca8108db0 0x7efca81a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 >> 172.21.15.107:0/504668400 conn(0x7efca8077910 msgr2=0x7efca8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.975+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 shutdown_connections 2024-03-11T19:50:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:40.976+0000 7efcacc00700 1 -- 172.21.15.107:0/504668400 wait complete. 2024-03-11T19:50:41.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:41 smithi107 ceph-mon[31712]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-11T19:50:41.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:41 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/504668400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:50:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:41 smithi122 ceph-mon[36397]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-11T19:50:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:41 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/504668400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:50:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:41 smithi145 ceph-mon[36730]: pgmap v343: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-11T19:50:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:41 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/504668400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:50:41.828 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1546798510},{"type":"v1","addr":"172.21.15.107:6801","nonce":1546798510}]},"active_addr":"172.21.15.107:6801/1546798510","active_change":"2024-03-11T19:41:54.175412+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":247446141}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3157443745}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3959959887}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2142594075}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3533566746}]}]} 2024-03-11T19:50:41.833 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-11T19:50:41.833 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-11T19:50:41.833 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:50:42.325 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:43.759 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:43 smithi107 ceph-mon[31712]: pgmap v344: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-03-11T19:50:43.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:43 smithi122 ceph-mon[36397]: pgmap v344: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-03-11T19:50:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:43 smithi145 ceph-mon[36730]: pgmap v344: 129 pgs: 129 active+clean; 626 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-03-11T19:50:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 -- 172.21.15.107:0/3511681912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34073ea0 msgr2=0x7fac34074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 --2- 172.21.15.107:0/3511681912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34073ea0 0x7fac34074280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac24009990 tx=0x7fac24009ca0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 -- 172.21.15.107:0/3511681912 shutdown_connections 2024-03-11T19:50:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 --2- 172.21.15.107:0/3511681912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac34112e40 0x7fac34115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 --2- 172.21.15.107:0/3511681912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34073ea0 0x7fac34074280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 --2- 172.21.15.107:0/3511681912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac340747c0 0x7fac34112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 -- 172.21.15.107:0/3511681912 >> 172.21.15.107:0/3511681912 conn(0x7fac34078dd0 msgr2=0x7fac340791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 -- 172.21.15.107:0/3511681912 shutdown_connections 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.532+0000 7fac39800700 1 -- 172.21.15.107:0/3511681912 wait complete. 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.533+0000 7fac39800700 1 Processor -- start 2024-03-11T19:50:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 -- start start 2024-03-11T19:50:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 0x7fac341a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac340747c0 0x7fac341a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3419ecb0 con 0x7fac34112e40 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3419edf0 con 0x7fac34073ea0 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac39800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3419efb0 con 0x7fac340747c0 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac33400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac33400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38034/0 (socket says 172.21.15.107:38034) 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac33400700 1 -- 172.21.15.107:0/1016930512 learned_addr learned my addr 172.21.15.107:0/1016930512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac2be00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac340747c0 0x7fac341a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.534+0000 7fac32a00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 0x7fac341a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 -- 172.21.15.107:0/1016930512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac340747c0 msgr2=0x7fac341a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac340747c0 0x7fac341a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 -- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 msgr2=0x7fac3419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 -- 172.21.15.107:0/1016930512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009520 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac33400700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac32a00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 0x7fac341a4440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac24005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.535+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2401d070 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.536+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac2400b610 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.536+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240173f0 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.536+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac240095f0 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.536+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3419f4b0 con 0x7fac34073ea0 2024-03-11T19:50:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.537+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34068930 con 0x7fac34073ea0 2024-03-11T19:50:44.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.540+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fac2400fad0 con 0x7fac34073ea0 2024-03-11T19:50:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.541+0000 7fac2b400700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 0x7fac180a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.541+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fac240cecd0 con 0x7fac34073ea0 2024-03-11T19:50:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.542+0000 7fac2be00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 0x7fac180a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:44.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.542+0000 7fac2be00700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 0x7fac180a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac34114cb0 tx=0x7fac1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.545+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac24097c80 con 0x7fac34073ea0 2024-03-11T19:50:44.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.754+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac34118390 con 0x7fac34073ea0 2024-03-11T19:50:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.756+0000 7fac2b400700 1 -- 172.21.15.107:0/1016930512 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7fac24027090 con 0x7fac34073ea0 2024-03-11T19:50:44.761 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:44.761 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:44.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 msgr2=0x7fac180a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:44.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 0x7fac180a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac34114cb0 tx=0x7fac1c009260 comp rx=0 tx=0).stop 2024-03-11T19:50:44.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 msgr2=0x7fac341a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:44.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 0x7fac341a4440 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac24005740 comp rx=0 tx=0).stop 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 shutdown_connections 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fac180a1b50 0x7fac180a4010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34112e40 0x7fac3419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.759+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac34073ea0 0x7fac341a4440 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.760+0000 7fac39800700 1 --2- 172.21.15.107:0/1016930512 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac340747c0 0x7fac341a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.760+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 >> 172.21.15.107:0/1016930512 conn(0x7fac34078dd0 msgr2=0x7fac34111c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.760+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 shutdown_connections 2024-03-11T19:50:44.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:44.760+0000 7fac39800700 1 -- 172.21.15.107:0/1016930512 wait complete. 2024-03-11T19:50:45.608 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-11T19:50:45.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:50:45.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:45 smithi107 ceph-mon[31712]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-11T19:50:45.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:45 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1016930512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:45.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:45 smithi122 ceph-mon[36397]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-11T19:50:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:45 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1016930512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:45 smithi145 ceph-mon[36730]: pgmap v345: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-03-11T19:50:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:45 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1016930512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:46.042 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:47.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:47 smithi122 ceph-mon[36397]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:50:47.770 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:47 smithi107 ceph-mon[31712]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:50:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:47 smithi145 ceph-mon[36730]: pgmap v346: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 -- 172.21.15.107:0/2562805550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff564104a70 msgr2=0x7ff564108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 --2- 172.21.15.107:0/2562805550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff564104a70 0x7ff564108ac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff550009990 tx=0x7ff550009ca0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 -- 172.21.15.107:0/2562805550 shutdown_connections 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 --2- 172.21.15.107:0/2562805550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff5641091a0 0x7ff564110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 --2- 172.21.15.107:0/2562805550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641040c0 0x7ff5641044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 --2- 172.21.15.107:0/2562805550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff564104a70 0x7ff564108ac0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 -- 172.21.15.107:0/2562805550 >> 172.21.15.107:0/2562805550 conn(0x7ff5640ff950 msgr2=0x7ff564101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:48.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 -- 172.21.15.107:0/2562805550 shutdown_connections 2024-03-11T19:50:48.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.331+0000 7ff56a000700 1 -- 172.21.15.107:0/2562805550 wait complete. 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.332+0000 7ff56a000700 1 Processor -- start 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.332+0000 7ff56a000700 1 -- start start 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641091a0 0x7ff56419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f270 con 0x7ff5641040c0 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f3e0 con 0x7ff5641091a0 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff56a000700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f5a0 con 0x7ff564104a70 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff563400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:48.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48738/0 (socket says 172.21.15.107:48738) 2024-03-11T19:50:48.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff563400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34536/0 (socket says 172.21.15.107:34536) 2024-03-11T19:50:48.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 -- 172.21.15.107:0/1760418394 learned_addr learned my addr 172.21.15.107:0/1760418394 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:50:48.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff563e00700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641091a0 0x7ff56419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 -- 172.21.15.107:0/1760418394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641091a0 msgr2=0x7ff56419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641091a0 0x7ff56419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 -- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 msgr2=0x7ff56419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff562a00700 1 -- 172.21.15.107:0/1760418394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5500095f0 con 0x7ff564104a70 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.333+0000 7ff563400700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff562a00700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff550000c00 tx=0x7ff550004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55001d070 con 0x7ff564104a70 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56419f820 con 0x7ff564104a70 2024-03-11T19:50:48.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff55000b610 con 0x7ff564104a70 2024-03-11T19:50:48.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500173f0 con 0x7ff564104a70 2024-03-11T19:50:48.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.334+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a70e0 con 0x7ff564104a70 2024-03-11T19:50:48.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.335+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56410c4c0 con 0x7ff564104a70 2024-03-11T19:50:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.338+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7ff55000fcb0 con 0x7ff564104a70 2024-03-11T19:50:48.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.339+0000 7ff55be00700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 0x7ff5480a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:48.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.339+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff5500cf110 con 0x7ff564104a70 2024-03-11T19:50:48.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.339+0000 7ff563400700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 0x7ff5480a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:48.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.340+0000 7ff563400700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 0x7ff5480a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff554005950 tx=0x7ff55400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:48.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.341+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff550097fb0 con 0x7ff564104a70 2024-03-11T19:50:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.570+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff56419b6f0 con 0x7ff564104a70 2024-03-11T19:50:48.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.572+0000 7ff55be00700 1 -- 172.21.15.107:0/1760418394 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24220 (secure 0 0 0) 0x7ff55009b670 con 0x7ff564104a70 2024-03-11T19:50:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:50:48.576 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":90,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:50:00.914592+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:50:48.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.574+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 msgr2=0x7ff5480a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.574+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 0x7ff5480a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff554005950 tx=0x7ff55400b410 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 msgr2=0x7ff56419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff550000c00 tx=0x7ff550004730 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 shutdown_connections 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff5480a1bd0 0x7ff5480a4090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641091a0 0x7ff56419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5641040c0 0x7ff56419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 --2- 172.21.15.107:0/1760418394 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff564104a70 0x7ff56419a8d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 >> 172.21.15.107:0/1760418394 conn(0x7ff5640ff950 msgr2=0x7ff564100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 shutdown_connections 2024-03-11T19:50:48.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:50:48.575+0000 7ff56a000700 1 -- 172.21.15.107:0/1760418394 wait complete. 2024-03-11T19:50:49.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-11T19:50:49.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-11T19:50:49.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-11T19:50:49.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-11T19:50:49.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-11T19:50:49.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-11T19:50:49.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-11T19:50:49.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-11T19:50:49.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-11T19:50:49.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-11T19:50:49.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-11T19:50:49.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-11T19:50:49.622 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:49 smithi107 ceph-mon[31712]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-11T19:50:49.622 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:49 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1760418394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:49 smithi122 ceph-mon[36397]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-11T19:50:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:49 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1760418394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:49 smithi145 ceph-mon[36730]: pgmap v347: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-11T19:50:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:49 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1760418394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:50:50.430 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:50.449 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:50.451 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:50.459 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:50.780 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:51.213 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:51.213 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:51.214 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:51.214 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:51 smithi122 ceph-mon[36397]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-11T19:50:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:51 smithi145 ceph-mon[36730]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-11T19:50:51.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:51 smithi107 ceph-mon[31712]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-11T19:50:53.707 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:53 smithi107 ceph-mon[31712]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-11T19:50:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:53 smithi122 ceph-mon[36397]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-11T19:50:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:53 smithi145 ceph-mon[36730]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-11T19:50:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-03-11T19:50:54.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-11T19:50:55.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:50:55.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-11T19:50:55.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:55 smithi107 ceph-mon[31712]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-11T19:50:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-11T19:50:55.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:50:55.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-11T19:50:55.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-11T19:50:55.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:55 smithi122 ceph-mon[36397]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:50:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-11T19:50:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-11T19:50:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-11T19:50:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:55 smithi145 ceph-mon[36730]: osdmap e91: 12 total, 12 up, 12 in 2024-03-11T19:50:56.147 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:56.151 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:50:56.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:56 smithi107 ceph-mon[31712]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:50:56.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:56 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:56.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:56 smithi122 ceph-mon[36397]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:50:56.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:56 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:56.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:56 smithi145 ceph-mon[36730]: osdmap e92: 12 total, 12 up, 12 in 2024-03-11T19:50:56.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:56 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:57 smithi122 ceph-mon[36397]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s 2024-03-11T19:50:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:57 smithi145 ceph-mon[36730]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s 2024-03-11T19:50:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:57 smithi107 ceph-mon[31712]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s 2024-03-11T19:50:59.159 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:59 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:50:59 smithi107 ceph-mon[31712]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-11T19:50:59.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:59 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:50:59 smithi122 ceph-mon[36397]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-11T19:50:59.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:59 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:50:59.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:50:59 smithi145 ceph-mon[36730]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-11T19:51:01.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:01 smithi107 ceph-mon[31712]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 6 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:01 smithi122 ceph-mon[36397]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 6 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:01 smithi145 ceph-mon[36730]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 6 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 -- 172.21.15.107:0/21123353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8101da0 msgr2=0x7fc1f8102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 --2- 172.21.15.107:0/21123353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8101da0 0x7fc1f8102180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009990 tx=0x7fc1e8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 -- 172.21.15.107:0/21123353 shutdown_connections 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 --2- 172.21.15.107:0/21123353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8106e80 0x7fc1f810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 --2- 172.21.15.107:0/21123353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8101da0 0x7fc1f8102180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 --2- 172.21.15.107:0/21123353 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f8102750 0x7fc1f81067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 -- 172.21.15.107:0/21123353 >> 172.21.15.107:0/21123353 conn(0x7fc1f80fd690 msgr2=0x7fc1f80ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 -- 172.21.15.107:0/21123353 shutdown_connections 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.254+0000 7fc1f7200700 1 -- 172.21.15.107:0/21123353 wait complete. 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.255+0000 7fc1f7200700 1 Processor -- start 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 -- start start 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 0x7fc1f8073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8102750 0x7fc1f80744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f8106e80 0x7fc1f80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a64f0 con 0x7fc1f8102750 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a6660 con 0x7fc1f8101da0 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.256+0000 7fc1f7200700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81a6820 con 0x7fc1f8106e80 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8102750 0x7fc1f80744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 0x7fc1f8073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:02.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f6800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f8106e80 0x7fc1f80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 0x7fc1f8073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43148/0 (socket says 172.21.15.107:43148) 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 -- 172.21.15.107:0/844715270 learned_addr learned my addr 172.21.15.107:0/844715270 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 -- 172.21.15.107:0/844715270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f8106e80 msgr2=0x7fc1f80749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1f8106e80 0x7fc1f80749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 -- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8102750 msgr2=0x7fc1f80744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f8102750 0x7fc1f80744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80095f0 con 0x7fc1f8101da0 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.257+0000 7fc1f5e00700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 0x7fc1f8073f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.258+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e801d070 con 0x7fc1f8101da0 2024-03-11T19:51:02.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.258+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1e800b610 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.258+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f81a6aa0 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.258+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80173f0 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.258+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f81a6eb0 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.259+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1f8111e80 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.261+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fc1e800fb00 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.262+0000 7fc1eea00700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc1dc0a1b50 0x7fc1dc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.262+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc1e80cf650 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.262+0000 7fc1f5400700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc1dc0a1b50 0x7fc1dc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1eea00700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] conn(0x7fc1dc0accb0 0x7fc1dc0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1dc0af820 con 0x7fc1dc0accb0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e80cf9c0 con 0x7fc1f8101da0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1f5400700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc1dc0a1b50 0x7fc1dc0a4010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8108390 tx=0x7fc1e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1f6800700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] conn(0x7fc1dc0accb0 0x7fc1dc0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:02.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.263+0000 7fc1f6800700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] conn(0x7fc1dc0accb0 0x7fc1dc0af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.264+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== osd.1 v2:172.21.15.122:6800/4246700402 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc1dc0af820 con 0x7fc1dc0accb0 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.282+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 --> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1f8062380 con 0x7fc1dc0accb0 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.283+0000 7fc1eea00700 1 -- 172.21.15.107:0/844715270 <== osd.1 v2:172.21.15.122:6800/4246700402 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1f8062380 con 0x7fc1dc0accb0 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.283+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] conn(0x7fc1dc0accb0 msgr2=0x7fc1dc0af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.283+0000 7fc1f7200700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] conn(0x7fc1dc0accb0 0x7fc1dc0af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc1dc0a1b50 msgr2=0x7fc1dc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc1dc0a1b50 0x7fc1dc0a4010 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8108390 tx=0x7fc1e4009260 comp rx=0 tx=0).stop 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 msgr2=0x7fc1f8073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 --2- 172.21.15.107:0/844715270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1f8101da0 0x7fc1f8073f70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e8005740 comp rx=0 tx=0).stop 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f6800700 1 -- 172.21.15.107:0/844715270 reap_dead start 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 shutdown_connections 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 >> 172.21.15.107:0/844715270 conn(0x7fc1f80fd690 msgr2=0x7fc1f80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 shutdown_connections 2024-03-11T19:51:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:01.284+0000 7fc1f7200700 1 -- 172.21.15.107:0/844715270 wait complete. 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.060+0000 7f2386800700 1 -- 172.21.15.107:0/3489437785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2380073ea0 msgr2=0x7f2380074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.060+0000 7f2386800700 1 --2- 172.21.15.107:0/3489437785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2380073ea0 0x7f2380074280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2370009990 tx=0x7f2370009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 -- 172.21.15.107:0/3489437785 shutdown_connections 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 --2- 172.21.15.107:0/3489437785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380112e40 0x7f2380115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 --2- 172.21.15.107:0/3489437785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2380073ea0 0x7f2380074280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 --2- 172.21.15.107:0/3489437785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23800747c0 0x7f2380112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 -- 172.21.15.107:0/3489437785 >> 172.21.15.107:0/3489437785 conn(0x7f2380078dd0 msgr2=0x7f23800791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 -- 172.21.15.107:0/3489437785 shutdown_connections 2024-03-11T19:51:03.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.061+0000 7f2386800700 1 -- 172.21.15.107:0/3489437785 wait complete. 2024-03-11T19:51:03.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 Processor -- start 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 -- start start 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 0x7f23801a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23800747c0 0x7f23801a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2380112e40 0x7f238019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019ee60 con 0x7f2380112e40 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019efd0 con 0x7f2380073ea0 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.062+0000 7f2386800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019f190 con 0x7f23800747c0 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 0x7f23801a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 0x7f23801a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43192/0 (socket says 172.21.15.107:43192) 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 -- 172.21.15.107:0/1232550383 learned_addr learned my addr 172.21.15.107:0/1232550383 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237ea00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23800747c0 0x7f23801a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:03.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237fe00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2380112e40 0x7f238019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 -- 172.21.15.107:0/1232550383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23800747c0 msgr2=0x7f23801a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23800747c0 0x7f23801a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 -- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2380112e40 msgr2=0x7f238019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2380112e40 0x7f238019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700095f0 con 0x7f2380073ea0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237ea00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23800747c0 0x7f23801a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.063+0000 7f237f400700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 0x7f23801a45e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2370000c00 tx=0x7f2370004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.064+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237001d070 con 0x7f2380073ea0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.064+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f237000b610 con 0x7f2380073ea0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.064+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238019f410 con 0x7f2380073ea0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.064+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700173f0 con 0x7f2380073ea0 2024-03-11T19:51:03.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.064+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23801ab560 con 0x7f2380073ea0 2024-03-11T19:51:03.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.065+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2380118660 con 0x7f2380073ea0 2024-03-11T19:51:03.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.067+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2370022670 con 0x7f2380073ea0 2024-03-11T19:51:03.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.068+0000 7f2377e00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23640a1b50 0x7f23640a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:03.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.068+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f23700cf120 con 0x7f2380073ea0 2024-03-11T19:51:03.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f237ea00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23640a1b50 0x7f23640a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:03.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f2377e00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] conn(0x7f23640accb0 0x7f23640af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:03.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23640af820 con 0x7f23640accb0 2024-03-11T19:51:03.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23700cf490 con 0x7f2380073ea0 2024-03-11T19:51:03.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f237fe00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] conn(0x7f23640accb0 0x7f23640af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:03.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.069+0000 7f237ea00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23640a1b50 0x7f23640a4010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f238019fe70 tx=0x7f236c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:03.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.070+0000 7f237fe00700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] conn(0x7f23640accb0 0x7f23640af150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:03.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.071+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== osd.3 v2:172.21.15.107:6810/3455904746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23640af820 con 0x7f23640accb0 2024-03-11T19:51:03.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.089+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 --> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f23801124f0 con 0x7f23640accb0 2024-03-11T19:51:03.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.089+0000 7f2377e00700 1 -- 172.21.15.107:0/1232550383 <== osd.3 v2:172.21.15.107:6810/3455904746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f23801124f0 con 0x7f23640accb0 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] conn(0x7f23640accb0 msgr2=0x7f23640af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] conn(0x7f23640accb0 0x7f23640af150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23640a1b50 msgr2=0x7f23640a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f23640a1b50 0x7f23640a4010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f238019fe70 tx=0x7f236c009260 comp rx=0 tx=0).stop 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 msgr2=0x7f23801a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 --2- 172.21.15.107:0/1232550383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2380073ea0 0x7f23801a45e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2370000c00 tx=0x7f2370004730 comp rx=0 tx=0).stop 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f237fe00700 1 -- 172.21.15.107:0/1232550383 reap_dead start 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 shutdown_connections 2024-03-11T19:51:03.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 >> 172.21.15.107:0/1232550383 conn(0x7f2380078dd0 msgr2=0x7f2380111b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:03.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 shutdown_connections 2024-03-11T19:51:03.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:03.090+0000 7f2386800700 1 -- 172.21.15.107:0/1232550383 wait complete. 2024-03-11T19:51:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:03 smithi122 ceph-mon[36397]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:03 smithi145 ceph-mon[36730]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:03 smithi107 ceph-mon[31712]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 157 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:04.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 -- 172.21.15.107:0/574757808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9318107cc0 msgr2=0x7f93181080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 --2- 172.21.15.107:0/574757808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9318107cc0 0x7f93181080a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f930c0099c0 tx=0x7f930c009cd0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 -- 172.21.15.107:0/574757808 shutdown_connections 2024-03-11T19:51:04.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 --2- 172.21.15.107:0/574757808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f931806a2c0 0x7f931810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 --2- 172.21.15.107:0/574757808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9318107cc0 0x7f93181080a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 --2- 172.21.15.107:0/574757808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 -- 172.21.15.107:0/574757808 >> 172.21.15.107:0/574757808 conn(0x7f9318077aa0 msgr2=0x7f9318077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 -- 172.21.15.107:0/574757808 shutdown_connections 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.478+0000 7f931f400700 1 -- 172.21.15.107:0/574757808 wait complete. 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.479+0000 7f931f400700 1 Processor -- start 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.479+0000 7f931f400700 1 -- start start 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931806a2c0 0x7f9318101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9318107cc0 0x7f93181016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181a6590 con 0x7f931806a2c0 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181a6700 con 0x7f9318069920 2024-03-11T19:51:04.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931f400700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181a68c0 con 0x7f9318107cc0 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931ca00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931ca00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43166/0 (socket says 172.21.15.107:43166) 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931ca00700 1 -- 172.21.15.107:0/2404057567 learned_addr learned my addr 172.21.15.107:0/2404057567 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f9317e00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931806a2c0 0x7f9318101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.480+0000 7f931d400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9318107cc0 0x7f93181016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 -- 172.21.15.107:0/2404057567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9318107cc0 msgr2=0x7f93181016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9318107cc0 0x7f93181016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 -- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931806a2c0 msgr2=0x7f9318101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931806a2c0 0x7f9318101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9308009520 con 0x7f9318069920 2024-03-11T19:51:04.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931ca00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318100c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f930c005910 tx=0x7f930c005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c01d070 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.481+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c0095f0 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.482+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f930c003680 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.482+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00fa30 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.482+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93181a6e20 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.482+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9318111f40 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.486+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f930c00bd00 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.487+0000 7f9315600700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93000a1b50 0x7f93000a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f930c0ceda0 con 0x7f9318069920 2024-03-11T19:51:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f9317e00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93000a1b50 0x7f93000a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f9315600700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] conn(0x7f93000accb0 0x7f93000af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93000af820 con 0x7f93000accb0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930c0cf1b0 con 0x7f9318069920 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.488+0000 7f931d400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] conn(0x7f93000accb0 0x7f93000af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.489+0000 7f9317e00700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93000a1b50 0x7f93000a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9318078110 tx=0x7f9308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.489+0000 7f931d400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] conn(0x7f93000accb0 0x7f93000af150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.490+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== osd.0 v2:172.21.15.107:6802/2497501905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f93000af820 con 0x7f93000accb0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.507+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 --> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9318062380 con 0x7f93000accb0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.508+0000 7f9315600700 1 -- 172.21.15.107:0/2404057567 <== osd.0 v2:172.21.15.107:6802/2497501905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9318062380 con 0x7f93000accb0 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.508+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] conn(0x7f93000accb0 msgr2=0x7f93000af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.508+0000 7f931f400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] conn(0x7f93000accb0 0x7f93000af150 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93000a1b50 msgr2=0x7f93000a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f93000a1b50 0x7f93000a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9318078110 tx=0x7f9308009260 comp rx=0 tx=0).stop 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 msgr2=0x7f9318100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 --2- 172.21.15.107:0/2404057567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318069920 0x7f9318100c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f930c005910 tx=0x7f930c005280 comp rx=0 tx=0).stop 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931d400700 1 -- 172.21.15.107:0/2404057567 reap_dead start 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 shutdown_connections 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 >> 172.21.15.107:0/2404057567 conn(0x7f9318077aa0 msgr2=0x7f93180ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 shutdown_connections 2024-03-11T19:51:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.509+0000 7f931f400700 1 -- 172.21.15.107:0/2404057567 wait complete. 2024-03-11T19:51:05.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:05 smithi107 ceph-mon[31712]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 127 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:05.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:05 smithi122 ceph-mon[36397]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 127 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:05.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:05 smithi145 ceph-mon[36730]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 127 B/s, 1 keys/s, 1 objects/s recovering 2024-03-11T19:51:05.800 INFO:teuthology.orchestra.run.smithi107.stdout:55834574932 2024-03-11T19:51:05.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-11T19:51:07.391 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:07.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:07 smithi122 ceph-mon[36397]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 116 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:07 smithi145 ceph-mon[36730]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 116 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:07.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:07 smithi107 ceph-mon[31712]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 116 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:107374182473 2024-03-11T19:51:08.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-11T19:51:08.487 INFO:teuthology.orchestra.run.smithi107.stdout:34359738458 2024-03-11T19:51:08.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-11T19:51:08.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.425+0000 7f9291c00700 1 -- 172.21.15.107:0/1431199977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0740a0 msgr2=0x7f928c074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.425+0000 7f9291c00700 1 --2- 172.21.15.107:0/1431199977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0740a0 0x7f928c074480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f927c009990 tx=0x7f927c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 -- 172.21.15.107:0/1431199977 shutdown_connections 2024-03-11T19:51:08.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 --2- 172.21.15.107:0/1431199977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c10e390 0x7f928c110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 --2- 172.21.15.107:0/1431199977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0740a0 0x7f928c074480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 --2- 172.21.15.107:0/1431199977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f928c0749c0 0x7f928c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 -- 172.21.15.107:0/1431199977 >> 172.21.15.107:0/1431199977 conn(0x7f928c0fd640 msgr2=0x7f928c0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 -- 172.21.15.107:0/1431199977 shutdown_connections 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.426+0000 7f9291c00700 1 -- 172.21.15.107:0/1431199977 wait complete. 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.427+0000 7f9291c00700 1 Processor -- start 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.427+0000 7f9291c00700 1 -- start start 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.427+0000 7f9291c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 0x7f928c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.427+0000 7f9291c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0749c0 0x7f928c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.427+0000 7f9291c00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f928c10e390 0x7f928c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f9291c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c19f1f0 con 0x7f928c0740a0 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f9291c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c19f360 con 0x7f928c0749c0 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f9291c00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c19f520 con 0x7f928c10e390 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 0x7f928c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 0x7f928c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54754/0 (socket says 172.21.15.107:54754) 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 -- 172.21.15.107:0/409215741 learned_addr learned my addr 172.21.15.107:0/409215741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f9291200700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f928c10e390 0x7f928c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f9283e00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0749c0 0x7f928c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 -- 172.21.15.107:0/409215741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f928c10e390 msgr2=0x7f928c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f928c10e390 0x7f928c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 -- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0749c0 msgr2=0x7f928c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f928c0749c0 0x7f928c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.428+0000 7f928be00700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009520 con 0x7f928c0740a0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.429+0000 7f928be00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 0x7f928c19a360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f927c000c00 tx=0x7f927c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.429+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c01d070 con 0x7f928c0740a0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.429+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0095f0 con 0x7f928c0740a0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.429+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f927c00b610 con 0x7f928c0740a0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.429+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c1a6e90 con 0x7f928c0740a0 2024-03-11T19:51:08.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.430+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c02c440 con 0x7f928c0740a0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.430+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f928c114100 con 0x7f928c0740a0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.432+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f927c00fb00 con 0x7f928c0740a0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.433+0000 7f9289600700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f92700a1b50 0x7f92700a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f927c0cf340 con 0x7f928c0740a0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9283e00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f92700a1b50 0x7f92700a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9289600700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] conn(0x7f92700accb0 0x7f92700af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92700af820 con 0x7f92700accb0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f927c0cf6b0 con 0x7f928c0740a0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.434+0000 7f9291200700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] conn(0x7f92700accb0 0x7f92700af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.435+0000 7f9283e00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f92700a1b50 0x7f92700a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f928c10f640 tx=0x7f9274009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.435+0000 7f9291200700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] conn(0x7f92700accb0 0x7f92700af150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.436+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== osd.4 v2:172.21.15.122:6808/1080212908 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f92700af820 con 0x7f92700accb0 2024-03-11T19:51:08.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.454+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 --> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f928c062330 con 0x7f92700accb0 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.454+0000 7f9289600700 1 -- 172.21.15.107:0/409215741 <== osd.4 v2:172.21.15.122:6808/1080212908 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f928c062330 con 0x7f92700accb0 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] conn(0x7f92700accb0 msgr2=0x7f92700af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] conn(0x7f92700accb0 0x7f92700af150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f92700a1b50 msgr2=0x7f92700a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f92700a1b50 0x7f92700a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f928c10f640 tx=0x7f9274009260 comp rx=0 tx=0).stop 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 msgr2=0x7f928c19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 --2- 172.21.15.107:0/409215741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928c0740a0 0x7f928c19a360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f927c000c00 tx=0x7f927c005740 comp rx=0 tx=0).stop 2024-03-11T19:51:08.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291200700 1 -- 172.21.15.107:0/409215741 reap_dead start 2024-03-11T19:51:08.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 shutdown_connections 2024-03-11T19:51:08.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 >> 172.21.15.107:0/409215741 conn(0x7f928c0fd640 msgr2=0x7f928c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 shutdown_connections 2024-03-11T19:51:08.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:04.455+0000 7f9291c00700 1 -- 172.21.15.107:0/409215741 wait complete. 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.586+0000 7fdd7ae00700 1 -- 172.21.15.107:0/2742903568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 msgr2=0x7fdd7410ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.586+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/2742903568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7410ebc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00b0a0 tx=0x7fdd6c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 -- 172.21.15.107:0/2742903568 shutdown_connections 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/2742903568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7410ebc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/2742903568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd74101f20 0x7fdd74102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/2742903568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd741028d0 0x7fdd74106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 -- 172.21.15.107:0/2742903568 >> 172.21.15.107:0/2742903568 conn(0x7fdd74078dd0 msgr2=0x7fdd740791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 -- 172.21.15.107:0/2742903568 shutdown_connections 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.587+0000 7fdd7ae00700 1 -- 172.21.15.107:0/2742903568 wait complete. 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 Processor -- start 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 -- start start 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 0x7fdd7419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd741028d0 0x7fdd7419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419f3f0 con 0x7fdd74106ff0 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419f560 con 0x7fdd74101f20 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.588+0000 7fdd7ae00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419f720 con 0x7fdd741028d0 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 0x7fdd7419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 0x7fdd7419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43178/0 (socket says 172.21.15.107:43178) 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 -- 172.21.15.107:0/3586451719 learned_addr learned my addr 172.21.15.107:0/3586451719 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd78e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73400700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd741028d0 0x7fdd7419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 -- 172.21.15.107:0/3586451719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd741028d0 msgr2=0x7fdd7419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd741028d0 0x7fdd7419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 -- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 msgr2=0x7fdd7419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009d00 con 0x7fdd74101f20 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd78e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd74106ff0 0x7fdd7419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.589+0000 7fdd73e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 0x7fdd7419a510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d980 tx=0x7fdd6400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.590+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004220 con 0x7fdd74101f20 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.590+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7419fa00 con 0x7fdd74101f20 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.590+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd64007640 con 0x7fdd74101f20 2024-03-11T19:51:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.590+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64005720 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.590+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741a7260 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.591+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd74111ff0 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.593+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fdd6400b750 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.594+0000 7fdd70c00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdd580a1b50 0x7fdd580a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdd640cddf0 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd73400700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdd580a1b50 0x7fdd580a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd70c00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] conn(0x7fdd580accb0 0x7fdd580af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd580af820 con 0x7fdd580accb0 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd640ce160 con 0x7fdd74101f20 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd78e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] conn(0x7fdd580accb0 0x7fdd580af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.595+0000 7fdd73400700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdd580a1b50 0x7fdd580a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd74075f00 tx=0x7fdd60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.596+0000 7fdd78e00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] conn(0x7fdd580accb0 0x7fdd580af150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.597+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== osd.7 v2:172.21.15.122:6816/3856508746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdd580af820 con 0x7fdd580accb0 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.615+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 --> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd74100d90 con 0x7fdd580accb0 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd70c00700 1 -- 172.21.15.107:0/3586451719 <== osd.7 v2:172.21.15.122:6816/3856508746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdd74100d90 con 0x7fdd580accb0 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] conn(0x7fdd580accb0 msgr2=0x7fdd580af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] conn(0x7fdd580accb0 0x7fdd580af150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdd580a1b50 msgr2=0x7fdd580a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdd580a1b50 0x7fdd580a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdd74075f00 tx=0x7fdd60009260 comp rx=0 tx=0).stop 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 msgr2=0x7fdd7419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.616+0000 7fdd7ae00700 1 --2- 172.21.15.107:0/3586451719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd74101f20 0x7fdd7419a510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d980 tx=0x7fdd6400dc90 comp rx=0 tx=0).stop 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.617+0000 7fdd78e00700 1 -- 172.21.15.107:0/3586451719 reap_dead start 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.617+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 shutdown_connections 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.617+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 >> 172.21.15.107:0/3586451719 conn(0x7fdd74078dd0 msgr2=0x7fdd7410d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.617+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 shutdown_connections 2024-03-11T19:51:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:02.617+0000 7fdd7ae00700 1 -- 172.21.15.107:0/3586451719 wait complete. 2024-03-11T19:51:09.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:09 smithi122 ceph-mon[36397]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 105 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:09.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:09 smithi145 ceph-mon[36730]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 105 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:09 smithi107 ceph-mon[31712]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 105 B/s, 0 keys/s, 1 objects/s recovering 2024-03-11T19:51:11.042 INFO:teuthology.orchestra.run.smithi107.stdout:219043332145 2024-03-11T19:51:11.043 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-11T19:51:11.705 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:11 smithi107 ceph-mon[31712]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 103 B/s, 0 objects/s recovering 2024-03-11T19:51:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:11 smithi122 ceph-mon[36397]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 103 B/s, 0 objects/s recovering 2024-03-11T19:51:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:11 smithi145 ceph-mon[36730]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 103 B/s, 0 objects/s recovering 2024-03-11T19:51:11.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 -- 172.21.15.107:0/1476321232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 msgr2=0x7fb6c0115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 --2- 172.21.15.107:0/1476321232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c0115060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800b0a0 tx=0x7fb6b800b3b0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 -- 172.21.15.107:0/1476321232 shutdown_connections 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 --2- 172.21.15.107:0/1476321232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c0115060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 --2- 172.21.15.107:0/1476321232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108400 0x7fb6c01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 --2- 172.21.15.107:0/1476321232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108db0 0x7fb6c010cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 -- 172.21.15.107:0/1476321232 >> 172.21.15.107:0/1476321232 conn(0x7fb6c0077910 msgr2=0x7fb6c0079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 -- 172.21.15.107:0/1476321232 shutdown_connections 2024-03-11T19:51:11.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.987+0000 7fb6c5800700 1 -- 172.21.15.107:0/1476321232 wait complete. 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.988+0000 7fb6c5800700 1 Processor -- start 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 -- start start 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108400 0x7fb6c01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108db0 0x7fb6c01a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c019ec50 con 0x7fb6c010d490 2024-03-11T19:51:11.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c019ed90 con 0x7fb6c0108400 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c5800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c019ef50 con 0x7fb6c0108db0 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33864/0 (socket says 172.21.15.107:33864) 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6c4e00700 1 -- 172.21.15.107:0/839025712 learned_addr learned my addr 172.21.15.107:0/839025712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6bfe00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108400 0x7fb6c01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:11.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.989+0000 7fb6bf400700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108db0 0x7fb6c01a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 -- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108db0 msgr2=0x7fb6c01a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108db0 0x7fb6c01a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 -- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 msgr2=0x7fb6c019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b8009d00 con 0x7fb6c0108400 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6c4e00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6c010d490 0x7fb6c019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bf400700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb6c0108db0 0x7fb6c01a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bfe00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108400 0x7fb6c01a43e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9f0 tx=0x7fb6b000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.990+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000c9c0 con 0x7fb6c0108400 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.991+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c019f230 con 0x7fb6c0108400 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.991+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6b0010430 con 0x7fb6c0108400 2024-03-11T19:51:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.991+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0018660 con 0x7fb6c0108400 2024-03-11T19:51:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.991+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c01ab480 con 0x7fb6c0108400 2024-03-11T19:51:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.992+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6c0109890 con 0x7fb6c0108400 2024-03-11T19:51:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.994+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fb6b0010910 con 0x7fb6c0108400 2024-03-11T19:51:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bcc00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb6a40a1b50 0x7fb6a40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb6b0014070 con 0x7fb6c0108400 2024-03-11T19:51:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bf400700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb6a40a1b50 0x7fb6a40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bcc00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] conn(0x7fb6a40accb0 0x7fb6a40af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6a40af820 con 0x7fb6a40accb0 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.995+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b00ce460 con 0x7fb6c0108400 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.996+0000 7fb6c4e00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] conn(0x7fb6a40accb0 0x7fb6a40af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.996+0000 7fb6bf400700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb6a40a1b50 0x7fb6a40a4010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0107560 tx=0x7fb6ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.996+0000 7fb6c4e00700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] conn(0x7fb6a40accb0 0x7fb6a40af150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:11.997+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== osd.5 v2:172.21.15.145:6808/3574999361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb6a40af820 con 0x7fb6a40accb0 2024-03-11T19:51:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.016+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 --> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb6c0118500 con 0x7fb6a40accb0 2024-03-11T19:51:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.016+0000 7fb6bcc00700 1 -- 172.21.15.107:0/839025712 <== osd.5 v2:172.21.15.145:6808/3574999361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb6c0118500 con 0x7fb6a40accb0 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] conn(0x7fb6a40accb0 msgr2=0x7fb6a40af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] conn(0x7fb6a40accb0 0x7fb6a40af150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb6a40a1b50 msgr2=0x7fb6a40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb6a40a1b50 0x7fb6a40a4010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0107560 tx=0x7fb6ac009260 comp rx=0 tx=0).stop 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108400 msgr2=0x7fb6c01a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 --2- 172.21.15.107:0/839025712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c0108400 0x7fb6c01a43e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e9f0 tx=0x7fb6b000ed00 comp rx=0 tx=0).stop 2024-03-11T19:51:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c4e00700 1 -- 172.21.15.107:0/839025712 reap_dead start 2024-03-11T19:51:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 shutdown_connections 2024-03-11T19:51:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 >> 172.21.15.107:0/839025712 conn(0x7fb6c0077910 msgr2=0x7fb6c0079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.017+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 shutdown_connections 2024-03-11T19:51:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.018+0000 7fb6c5800700 1 -- 172.21.15.107:0/839025712 wait complete. 2024-03-11T19:51:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 -- 172.21.15.107:0/578102835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead4106fd0 msgr2=0x7fead410eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 --2- 172.21.15.107:0/578102835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead4106fd0 0x7fead410eba0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feacc009990 tx=0x7feacc009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 -- 172.21.15.107:0/578102835 shutdown_connections 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 --2- 172.21.15.107:0/578102835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead4106fd0 0x7fead410eba0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 --2- 172.21.15.107:0/578102835 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead4101ef0 0x7fead41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 --2- 172.21.15.107:0/578102835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead41028a0 0x7fead41068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.219+0000 7feadba00700 1 -- 172.21.15.107:0/578102835 >> 172.21.15.107:0/578102835 conn(0x7fead40fd740 msgr2=0x7fead40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.220+0000 7feadba00700 1 -- 172.21.15.107:0/578102835 shutdown_connections 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.220+0000 7feadba00700 1 -- 172.21.15.107:0/578102835 wait complete. 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.221+0000 7feadba00700 1 Processor -- start 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.221+0000 7feadba00700 1 -- start start 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 0x7fead419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead41028a0 0x7fead419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead4106fd0 0x7fead419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419f2a0 con 0x7fead41028a0 2024-03-11T19:51:12.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419f410 con 0x7fead4101ef0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feadba00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead419f5d0 con 0x7fead4106fd0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feacbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead41028a0 0x7fead419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 0x7fead419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 0x7fead419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:50860/0 (socket says 172.21.15.107:50860) 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 -- 172.21.15.107:0/3606176741 learned_addr learned my addr 172.21.15.107:0/3606176741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7feacbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead41028a0 0x7fead419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33876/0 (socket says 172.21.15.107:33876) 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9a00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead4106fd0 0x7fead419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 -- 172.21.15.107:0/3606176741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead4106fd0 msgr2=0x7fead419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fead4106fd0 0x7fead419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 -- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead41028a0 msgr2=0x7fead419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fead41028a0 0x7fead419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.222+0000 7fead9000700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc009520 con 0x7fead4101ef0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7fead9000700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 0x7fead419a3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feac400ef10 tx=0x7feac400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:12.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac400c9c0 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feac400f040 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0095f0 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4014480 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.223+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead41a6f40 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.224+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fead4112080 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.226+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7feac4007720 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.227+0000 7fead2000700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7feab80a1bd0 0x7feab80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7feac40ce630 con 0x7fead4101ef0 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7fead2000700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] conn(0x7feab80acd30 0x7feab80af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feab80af8a0 con 0x7feab80acd30 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7feacbe00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7feab80a1bd0 0x7feab80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7feac40ce9a0 con 0x7fead4101ef0 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.228+0000 7fead9a00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] conn(0x7feab80acd30 0x7feab80af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.229+0000 7feacbe00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7feab80a1bd0 0x7feab80a4090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fead410f050 tx=0x7feabc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.229+0000 7fead9a00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] conn(0x7feab80acd30 0x7feab80af1d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.230+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== osd.10 v2:172.21.15.122:6824/1130634331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7feab80af8a0 con 0x7feab80acd30 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.247+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 --> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fead40fe9b0 con 0x7feab80acd30 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.248+0000 7fead2000700 1 -- 172.21.15.107:0/3606176741 <== osd.10 v2:172.21.15.122:6824/1130634331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fead40fe9b0 con 0x7feab80acd30 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.248+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] conn(0x7feab80acd30 msgr2=0x7feab80af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.248+0000 7feadba00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] conn(0x7feab80acd30 0x7feab80af1d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7feab80a1bd0 msgr2=0x7feab80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7feab80a1bd0 0x7feab80a4090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fead410f050 tx=0x7feabc009260 comp rx=0 tx=0).stop 2024-03-11T19:51:12.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 msgr2=0x7fead419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 --2- 172.21.15.107:0/3606176741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead4101ef0 0x7fead419a3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feac400ef10 tx=0x7feac400c3a0 comp rx=0 tx=0).stop 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7fead9a00700 1 -- 172.21.15.107:0/3606176741 reap_dead start 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 shutdown_connections 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 >> 172.21.15.107:0/3606176741 conn(0x7fead40fd740 msgr2=0x7fead40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 shutdown_connections 2024-03-11T19:51:12.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:12.249+0000 7feadba00700 1 -- 172.21.15.107:0/3606176741 wait complete. 2024-03-11T19:51:13.206 INFO:teuthology.orchestra.run.smithi107.stdout:141733920836 2024-03-11T19:51:13.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-11T19:51:13.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 -- 172.21.15.107:0/382555530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac108e90 msgr2=0x7f2cac10cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/382555530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac108e90 0x7f2cac10cee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009990 tx=0x7f2c94009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 -- 172.21.15.107:0/382555530 shutdown_connections 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/382555530 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac10d5c0 0x7f2cac115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/382555530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac1084e0 0x7f2cac1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/382555530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac108e90 0x7f2cac10cee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 -- 172.21.15.107:0/382555530 >> 172.21.15.107:0/382555530 conn(0x7f2cac075440 msgr2=0x7f2cac077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 -- 172.21.15.107:0/382555530 shutdown_connections 2024-03-11T19:51:13.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.427+0000 7f2cb1e00700 1 -- 172.21.15.107:0/382555530 wait complete. 2024-03-11T19:51:13.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.428+0000 7f2cb1e00700 1 Processor -- start 2024-03-11T19:51:13.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 -- start start 2024-03-11T19:51:13.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac1084e0 0x7f2cac1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 0x7f2cac1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac10d5c0 0x7f2cac19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19ec60 con 0x7f2cac108e90 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19eda0 con 0x7f2cac1084e0 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2cb1e00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19ef60 con 0x7f2cac10d5c0 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2ca2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 0x7f2cac1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2ca2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 0x7f2cac1a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33892/0 (socket says 172.21.15.107:33892) 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2caaa00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac1084e0 0x7f2cac1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.429+0000 7f2ca2000700 1 -- 172.21.15.107:0/1605322160 learned_addr learned my addr 172.21.15.107:0/1605322160 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:13.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2cab400700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac10d5c0 0x7f2cac19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:13.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 -- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac10d5c0 msgr2=0x7f2cac19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac10d5c0 0x7f2cac19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 -- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 msgr2=0x7f2cac1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 0x7f2cac1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940095f0 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2ca2000700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac108e90 0x7f2cac1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.430+0000 7f2caaa00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac1084e0 0x7f2cac1a43f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9f0 tx=0x7f2c9c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.431+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00c9c0 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.431+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c9c010430 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.431+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac19f240 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.431+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c018680 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.431+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1ab490 con 0x7f2cac1084e0 2024-03-11T19:51:13.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.432+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cac1099c0 con 0x7f2cac1084e0 2024-03-11T19:51:13.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.434+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2c9c010910 con 0x7f2cac1084e0 2024-03-11T19:51:13.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.436+0000 7f2cb0a00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c900a1bd0 0x7f2c900a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:13.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.436+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2c9c014070 con 0x7f2cac1084e0 2024-03-11T19:51:13.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.436+0000 7f2ca2000700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c900a1bd0 0x7f2c900a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:13.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.437+0000 7f2cb0a00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] conn(0x7f2c900acd30 0x7f2c900af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:13.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.437+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c900af8a0 con 0x7f2c900acd30 2024-03-11T19:51:13.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.437+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9c0cfe50 con 0x7f2cac1084e0 2024-03-11T19:51:13.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.437+0000 7f2ca2000700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c900a1bd0 0x7f2c900a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cac1074d0 tx=0x7f2c9400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:13.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.437+0000 7f2cab400700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] conn(0x7f2c900acd30 0x7f2c900af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:13.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.438+0000 7f2cab400700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] conn(0x7f2c900acd30 0x7f2c900af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:13.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.439+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== osd.8 v2:172.21.15.145:6816/4190884863 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c900af8a0 con 0x7f2c900acd30 2024-03-11T19:51:13.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.457+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 --> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2cac118650 con 0x7f2c900acd30 2024-03-11T19:51:13.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.457+0000 7f2cb0a00700 1 -- 172.21.15.107:0/1605322160 <== osd.8 v2:172.21.15.145:6816/4190884863 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2cac118650 con 0x7f2c900acd30 2024-03-11T19:51:13.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] conn(0x7f2c900acd30 msgr2=0x7f2c900af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] conn(0x7f2c900acd30 0x7f2c900af1d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:13.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c900a1bd0 msgr2=0x7f2c900a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c900a1bd0 0x7f2c900a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cac1074d0 tx=0x7f2c9400b280 comp rx=0 tx=0).stop 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac1084e0 msgr2=0x7f2cac1a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 --2- 172.21.15.107:0/1605322160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac1084e0 0x7f2cac1a43f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9f0 tx=0x7f2c9c00ed00 comp rx=0 tx=0).stop 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cab400700 1 -- 172.21.15.107:0/1605322160 reap_dead start 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 shutdown_connections 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.458+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 >> 172.21.15.107:0/1605322160 conn(0x7f2cac075440 msgr2=0x7f2cac077740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.459+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 shutdown_connections 2024-03-11T19:51:13.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:13.459+0000 7f2cb1e00700 1 -- 172.21.15.107:0/1605322160 wait complete. 2024-03-11T19:51:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:13 smithi122 ceph-mon[36397]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:13.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:13 smithi145 ceph-mon[36730]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:13.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:13 smithi107 ceph-mon[31712]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:14.530 INFO:teuthology.orchestra.run.smithi107.stdout:309237645346 2024-03-11T19:51:14.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-11T19:51:15.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:15 smithi107 ceph-mon[31712]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:15.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 -- 172.21.15.107:0/2634657021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028101f20 msgr2=0x7fc028102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 --2- 172.21.15.107:0/2634657021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028101f20 0x7fc028102300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc018009990 tx=0x7fc018009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 -- 172.21.15.107:0/2634657021 shutdown_connections 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 --2- 172.21.15.107:0/2634657021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc028106ff0 0x7fc02810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 --2- 172.21.15.107:0/2634657021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028101f20 0x7fc028102300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 --2- 172.21.15.107:0/2634657021 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0281028d0 0x7fc028106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 -- 172.21.15.107:0/2634657021 >> 172.21.15.107:0/2634657021 conn(0x7fc028078dd0 msgr2=0x7fc0280791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 -- 172.21.15.107:0/2634657021 shutdown_connections 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.026+0000 7fc02d200700 1 -- 172.21.15.107:0/2634657021 wait complete. 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.027+0000 7fc02d200700 1 Processor -- start 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 -- start start 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc028101f20 0x7fc028073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 0x7fc0280744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106ff0 0x7fc0280749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281a6660 con 0x7fc0281028d0 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281a67d0 con 0x7fc028106ff0 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc02d200700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281a6990 con 0x7fc028101f20 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.028+0000 7fc025400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 0x7fc0280744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 0x7fc0280744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45732/0 (socket says 172.21.15.107:45732) 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 -- 172.21.15.107:0/1438939044 learned_addr learned my addr 172.21.15.107:0/1438939044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc026800700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106ff0 0x7fc0280749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025e00700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc028101f20 0x7fc028073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 -- 172.21.15.107:0/1438939044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc028101f20 msgr2=0x7fc028073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc028101f20 0x7fc028073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 -- 172.21.15.107:0/1438939044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106ff0 msgr2=0x7fc0280749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106ff0 0x7fc0280749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180095f0 con 0x7fc0281028d0 2024-03-11T19:51:15.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.029+0000 7fc025400700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 0x7fc0280744b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc01400e970 tx=0x7fc01400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:15.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.030+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014009790 con 0x7fc0281028d0 2024-03-11T19:51:15.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.030+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0281a6c70 con 0x7fc0281028d0 2024-03-11T19:51:15.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.030+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc01400c7e0 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.030+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014005580 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.030+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0281a71c0 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.031+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc028111fe0 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.033+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fc01400cce0 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.034+0000 7fc01ea00700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc00c0a1bd0 0x7fc00c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.034+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc014014070 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.034+0000 7fc025e00700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc00c0a1bd0 0x7fc00c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.035+0000 7fc01ea00700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] conn(0x7fc00c0acd30 0x7fc00c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.035+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc00c0af8a0 con 0x7fc00c0acd30 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.035+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0140d22a0 con 0x7fc0281028d0 2024-03-11T19:51:15.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.035+0000 7fc026800700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] conn(0x7fc00c0acd30 0x7fc00c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.035+0000 7fc025e00700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc00c0a1bd0 0x7fc00c0a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b2e0 tx=0x7fc018005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.036+0000 7fc026800700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] conn(0x7fc00c0acd30 0x7fc00c0af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.037+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== osd.9 v2:172.21.15.107:6826/3311766141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc00c0af8a0 con 0x7fc00c0acd30 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.055+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 --> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc028062330 con 0x7fc00c0acd30 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.055+0000 7fc01ea00700 1 -- 172.21.15.107:0/1438939044 <== osd.9 v2:172.21.15.107:6826/3311766141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc028062330 con 0x7fc00c0acd30 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] conn(0x7fc00c0acd30 msgr2=0x7fc00c0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] conn(0x7fc00c0acd30 0x7fc00c0af1d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc00c0a1bd0 msgr2=0x7fc00c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc00c0a1bd0 0x7fc00c0a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b2e0 tx=0x7fc018005960 comp rx=0 tx=0).stop 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 msgr2=0x7fc0280744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 --2- 172.21.15.107:0/1438939044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0281028d0 0x7fc0280744b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc01400e970 tx=0x7fc01400ec80 comp rx=0 tx=0).stop 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc026800700 1 -- 172.21.15.107:0/1438939044 reap_dead start 2024-03-11T19:51:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 shutdown_connections 2024-03-11T19:51:15.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 >> 172.21.15.107:0/1438939044 conn(0x7fc028078dd0 msgr2=0x7fc028106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:15.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 shutdown_connections 2024-03-11T19:51:15.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.056+0000 7fc02d200700 1 -- 172.21.15.107:0/1438939044 wait complete. 2024-03-11T19:51:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:15 smithi122 ceph-mon[36397]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:15.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:15 smithi145 ceph-mon[36730]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:17.477 INFO:teuthology.orchestra.run.smithi107.stdout:287762808873 2024-03-11T19:51:17.477 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-11T19:51:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:17 smithi122 ceph-mon[36397]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:51:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:17 smithi145 ceph-mon[36730]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:51:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:17 smithi107 ceph-mon[31712]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.118+0000 7fdd18800700 1 -- 172.21.15.107:0/486723111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd10101f20 msgr2=0x7fdd10102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.118+0000 7fdd18800700 1 --2- 172.21.15.107:0/486723111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd10101f20 0x7fdd10102300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098f0 tx=0x7fdd04009c00 comp rx=0 tx=0).stop 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 -- 172.21.15.107:0/486723111 shutdown_connections 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 --2- 172.21.15.107:0/486723111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10106ff0 0x7fdd1010ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 --2- 172.21.15.107:0/486723111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd10101f20 0x7fdd10102300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 --2- 172.21.15.107:0/486723111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd101028d0 0x7fdd10106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 -- 172.21.15.107:0/486723111 >> 172.21.15.107:0/486723111 conn(0x7fdd10078dd0 msgr2=0x7fdd100791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 -- 172.21.15.107:0/486723111 shutdown_connections 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.119+0000 7fdd18800700 1 -- 172.21.15.107:0/486723111 wait complete. 2024-03-11T19:51:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.120+0000 7fdd18800700 1 Processor -- start 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.120+0000 7fdd18800700 1 -- start start 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 0x7fdd1019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd101028d0 0x7fdd1019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd10106ff0 0x7fdd1019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f390 con 0x7fdd10101f20 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f500 con 0x7fdd101028d0 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd18800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f6c0 con 0x7fdd10106ff0 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd15e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 0x7fdd1019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd15e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 0x7fdd1019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45768/0 (socket says 172.21.15.107:45768) 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd15e00700 1 -- 172.21.15.107:0/2757486542 learned_addr learned my addr 172.21.15.107:0/2757486542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd15400700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd101028d0 0x7fdd1019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.121+0000 7fdd16800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd10106ff0 0x7fdd1019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 -- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd10106ff0 msgr2=0x7fdd1019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd10106ff0 0x7fdd1019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 -- 172.21.15.107:0/2757486542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd101028d0 msgr2=0x7fdd1019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd101028d0 0x7fdd1019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04009520 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15400700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd101028d0 0x7fdd1019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd15e00700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 0x7fdd1019a4b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009230 tx=0x7fdd04005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0401c070 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1019f940 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd0400ba40 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003d20 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.122+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd101a7200 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.123+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd10112060 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.125+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fdd0400b560 con 0x7fdd10101f20 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd0aa00700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdcf80a1b50 0x7fdcf80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:18.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdd040cea40 con 0x7fdd10101f20 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd15400700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdcf80a1b50 0x7fdcf80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd0aa00700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] conn(0x7fdcf80accb0 0x7fdcf80af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcf80af820 con 0x7fdcf80accb0 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.127+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd040cedb0 con 0x7fdd10101f20 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.128+0000 7fdd15400700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdcf80a1b50 0x7fdcf80a4010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10101be0 tx=0x7fdd000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:18.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.128+0000 7fdd16800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] conn(0x7fdcf80accb0 0x7fdcf80af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.128+0000 7fdd16800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] conn(0x7fdcf80accb0 0x7fdcf80af150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.129+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== osd.11 v2:172.21.15.145:6824/2544565646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdcf80af820 con 0x7fdcf80accb0 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.147+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 --> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd1010e180 con 0x7fdcf80accb0 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.148+0000 7fdd0aa00700 1 -- 172.21.15.107:0/2757486542 <== osd.11 v2:172.21.15.145:6824/2544565646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdd1010e180 con 0x7fdcf80accb0 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.148+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] conn(0x7fdcf80accb0 msgr2=0x7fdcf80af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.148+0000 7fdd18800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] conn(0x7fdcf80accb0 0x7fdcf80af150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdcf80a1b50 msgr2=0x7fdcf80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdcf80a1b50 0x7fdcf80a4010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10101be0 tx=0x7fdd000073d0 comp rx=0 tx=0).stop 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 msgr2=0x7fdd1019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 --2- 172.21.15.107:0/2757486542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd10101f20 0x7fdd1019a4b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009230 tx=0x7fdd04005040 comp rx=0 tx=0).stop 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd16800700 1 -- 172.21.15.107:0/2757486542 reap_dead start 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 shutdown_connections 2024-03-11T19:51:18.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 >> 172.21.15.107:0/2757486542 conn(0x7fdd10078dd0 msgr2=0x7fdd10100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:18.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 shutdown_connections 2024-03-11T19:51:18.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:18.149+0000 7fdd18800700 1 -- 172.21.15.107:0/2757486542 wait complete. 2024-03-11T19:51:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 -- 172.21.15.107:0/3661418026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 msgr2=0x7f60e810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 --2- 172.21.15.107:0/3661418026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e810eb60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60e000b0a0 tx=0x7f60e000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 -- 172.21.15.107:0/3661418026 shutdown_connections 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 --2- 172.21.15.107:0/3661418026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e810eb60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 --2- 172.21.15.107:0/3661418026 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e806a570 0x7f60e806a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 --2- 172.21.15.107:0/3661418026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806af20 0x7f60e81068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 -- 172.21.15.107:0/3661418026 >> 172.21.15.107:0/3661418026 conn(0x7f60e8077ae0 msgr2=0x7f60e8077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 -- 172.21.15.107:0/3661418026 shutdown_connections 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.631+0000 7f60ee200700 1 -- 172.21.15.107:0/3661418026 wait complete. 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.632+0000 7f60ee200700 1 Processor -- start 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 -- start start 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 0x7f60e8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e806af20 0x7f60e8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e819d1d0 con 0x7f60e8106fe0 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e819d340 con 0x7f60e806a570 2024-03-11T19:51:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60ee200700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e819d500 con 0x7f60e806af20 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e7400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 0x7f60e8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45750/0 (socket says 172.21.15.107:45750) 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e7400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 0x7f60e8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:34800/0 (socket says 172.21.15.107:34800) 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e7e00700 1 -- 172.21.15.107:0/4160095211 learned_addr learned my addr 172.21.15.107:0/4160095211 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.633+0000 7f60e6a00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e806af20 0x7f60e8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 -- 172.21.15.107:0/4160095211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e806af20 msgr2=0x7f60e8198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60e806af20 0x7f60e8198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 -- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 msgr2=0x7f60e819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009d00 con 0x7f60e806a570 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7e00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e8106fe0 0x7f60e819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.634+0000 7f60e7400700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 0x7f60e8198340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9f0 tx=0x7f60d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.635+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800c9c0 con 0x7f60e806a570 2024-03-11T19:51:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.635+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d8010430 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.635+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e819d7e0 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.635+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8018680 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.635+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e81a5040 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.636+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60e8112040 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.639+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f60d8010910 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60dfe00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60cc0a1bd0 0x7f60cc0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f60d8014070 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60e6a00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60cc0a1bd0 0x7f60cc0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60dfe00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] conn(0x7f60cc0acd30 0x7f60cc0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60cc0af8a0 con 0x7f60cc0acd30 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.641+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d80cdd30 con 0x7f60e806a570 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.642+0000 7f60e7e00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] conn(0x7f60cc0acd30 0x7f60cc0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.642+0000 7f60e6a00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60cc0a1bd0 0x7f60cc0a4090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f60e81006f0 tx=0x7f60d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.642+0000 7f60e7e00700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] conn(0x7f60cc0acd30 0x7f60cc0af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.643+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== osd.6 v2:172.21.15.107:6818/1993322562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f60cc0af8a0 con 0x7f60cc0acd30 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.661+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 --> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f60e8062300 con 0x7f60cc0acd30 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.662+0000 7f60dfe00700 1 -- 172.21.15.107:0/4160095211 <== osd.6 v2:172.21.15.107:6818/1993322562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f60e8062300 con 0x7f60cc0acd30 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] conn(0x7f60cc0acd30 msgr2=0x7f60cc0af1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] conn(0x7f60cc0acd30 0x7f60cc0af1d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60cc0a1bd0 msgr2=0x7f60cc0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60cc0a1bd0 0x7f60cc0a4090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f60e81006f0 tx=0x7f60d4009260 comp rx=0 tx=0).stop 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 msgr2=0x7f60e8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 --2- 172.21.15.107:0/4160095211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e806a570 0x7f60e8198340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9f0 tx=0x7f60d800ed00 comp rx=0 tx=0).stop 2024-03-11T19:51:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60e7e00700 1 -- 172.21.15.107:0/4160095211 reap_dead start 2024-03-11T19:51:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 shutdown_connections 2024-03-11T19:51:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 >> 172.21.15.107:0/4160095211 conn(0x7f60e8077ae0 msgr2=0x7f60e80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 shutdown_connections 2024-03-11T19:51:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:15.663+0000 7f60ee200700 1 -- 172.21.15.107:0/4160095211 wait complete. 2024-03-11T19:51:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:19 smithi122 ceph-mon[36397]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:51:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:19 smithi145 ceph-mon[36730]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:51:19.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:19 smithi107 ceph-mon[31712]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-11T19:51:19.910 INFO:teuthology.orchestra.run.smithi107.stdout:163208757312 2024-03-11T19:51:19.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-11T19:51:20.631 INFO:teuthology.orchestra.run.smithi107.stdout:197568495674 2024-03-11T19:51:20.632 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-11T19:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 -- 172.21.15.107:0/813160917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 msgr2=0x7f045410eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 --2- 172.21.15.107:0/813160917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f045410eb30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f044c00b0d0 tx=0x7f044c00b3e0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 -- 172.21.15.107:0/813160917 shutdown_connections 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 --2- 172.21.15.107:0/813160917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f045410eb30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 --2- 172.21.15.107:0/813160917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0454107cf0 0x7f04541080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 --2- 172.21.15.107:0/813160917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f045406a370 0x7f045406a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 -- 172.21.15.107:0/813160917 >> 172.21.15.107:0/813160917 conn(0x7f0454076b90 msgr2=0x7f0454076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 -- 172.21.15.107:0/813160917 shutdown_connections 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.634+0000 7f0459e00700 1 -- 172.21.15.107:0/813160917 wait complete. 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.635+0000 7f0459e00700 1 Processor -- start 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.635+0000 7f0459e00700 1 -- start start 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f045406a370 0x7f0454198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f04541988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454107cf0 0x7f045419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045419d1f0 con 0x7f045406ad10 2024-03-11T19:51:20.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045419d360 con 0x7f045406a370 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0459e00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045419d520 con 0x7f0454107cf0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0452000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f04541988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0452000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f04541988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45778/0 (socket says 172.21.15.107:45778) 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0452000700 1 -- 172.21.15.107:0/4145906918 learned_addr learned my addr 172.21.15.107:0/4145906918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.636+0000 7f0452a00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f045406a370 0x7f0454198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454107cf0 0x7f045419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 -- 172.21.15.107:0/4145906918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f045406a370 msgr2=0x7f0454198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f045406a370 0x7f0454198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 -- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 msgr2=0x7f04541988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f04541988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c009d00 con 0x7f0454107cf0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0452000700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f045406ad10 0x7f04541988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.637+0000 7f0453400700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454107cf0 0x7f045419cbe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f044c00d010 tx=0x7f044c00bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:20.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.638+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00e040 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.638+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f044c005070 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.638+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045419d7a0 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.638+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c01d6a0 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.638+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04541a5060 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.639+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0454111f20 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.641+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f044c0051e0 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.642+0000 7f0458a00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f04380a1b50 0x7f04380a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.642+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f044c0cebb0 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0452a00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f04380a1b50 0x7f04380a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0458a00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] conn(0x7f04380acb80 0x7f04380af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04380af6f0 con 0x7f04380acb80 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f044c0cef20 con 0x7f0454107cf0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0452000700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] conn(0x7f04380acb80 0x7f04380af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:20.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.643+0000 7f0452a00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f04380a1b50 0x7f04380a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0444007b60 tx=0x7f04440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.644+0000 7f0452000700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] conn(0x7f04380acb80 0x7f04380af020 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.645+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== osd.2 v2:172.21.15.145:6800/3949304932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f04380af6f0 con 0x7f04380acb80 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.662+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 --> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0454062380 con 0x7f04380acb80 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.663+0000 7f0458a00700 1 -- 172.21.15.107:0/4145906918 <== osd.2 v2:172.21.15.145:6800/3949304932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0454062380 con 0x7f04380acb80 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] conn(0x7f04380acb80 msgr2=0x7f04380af020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] conn(0x7f04380acb80 0x7f04380af020 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f04380a1b50 msgr2=0x7f04380a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f04380a1b50 0x7f04380a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0444007b60 tx=0x7f04440073d0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454107cf0 msgr2=0x7f045419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 --2- 172.21.15.107:0/4145906918 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0454107cf0 0x7f045419cbe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f044c00d010 tx=0x7f044c00bdc0 comp rx=0 tx=0).stop 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0453400700 1 -- 172.21.15.107:0/4145906918 reap_dead start 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 shutdown_connections 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 >> 172.21.15.107:0/4145906918 conn(0x7f0454076b90 msgr2=0x7f04540ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 shutdown_connections 2024-03-11T19:51:20.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:20.664+0000 7f0459e00700 1 -- 172.21.15.107:0/4145906918 wait complete. 2024-03-11T19:51:21.196 INFO:teuthology.orchestra.run.smithi107.stdout:253403070511 2024-03-11T19:51:21.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-11T19:51:21.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:21 smithi107 ceph-mon[31712]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:21.714 INFO:teuthology.orchestra.run.smithi107.stdout:77309411411 2024-03-11T19:51:21.714 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-11T19:51:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:21 smithi122 ceph-mon[36397]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:21 smithi145 ceph-mon[36730]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:22.694 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:22.700 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:22.701 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:22.703 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:22.705 INFO:teuthology.orchestra.run.smithi107.stdout:343597383711 2024-03-11T19:51:22.705 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-11T19:51:22.802 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:23 smithi122 ceph-mon[36397]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:23.768 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:23 smithi107 ceph-mon[31712]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:23 smithi145 ceph-mon[36730]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:24.514 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:25.389 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:25.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:25 smithi107 ceph-mon[31712]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:51:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:25 smithi122 ceph-mon[36397]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:51:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:25 smithi145 ceph-mon[36730]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:51:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:27 smithi122 ceph-mon[36397]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:27 smithi145 ceph-mon[36730]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:27 smithi107 ceph-mon[31712]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:29.556 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:29 smithi122 ceph-mon[36397]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:29 smithi145 ceph-mon[36730]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:29 smithi107 ceph-mon[31712]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T19:51:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:31.552 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:31 smithi122 ceph-mon[36397]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-11T19:51:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:31 smithi145 ceph-mon[36730]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-11T19:51:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:31 smithi107 ceph-mon[31712]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-11T19:51:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:33 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:33 smithi122 ceph-mon[36397]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:33.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:33 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:33 smithi145 ceph-mon[36730]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:33.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:33 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:33.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:33 smithi107 ceph-mon[31712]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:34.328 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:34.328 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:34.328 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 -- 172.21.15.107:0/600094214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8108db0 msgr2=0x7f2bd810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/600094214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8108db0 0x7f2bd810cdb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009990 tx=0x7f2bc4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 -- 172.21.15.107:0/600094214 shutdown_connections 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/600094214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd810d490 0x7f2bd8115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/600094214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108400 0x7f2bd81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/600094214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd8108db0 0x7f2bd810cdb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 -- 172.21.15.107:0/600094214 >> 172.21.15.107:0/600094214 conn(0x7f2bd8077910 msgr2=0x7f2bd8079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 -- 172.21.15.107:0/600094214 shutdown_connections 2024-03-11T19:51:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.216+0000 7f2bd7e00700 1 -- 172.21.15.107:0/600094214 wait complete. 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.217+0000 7f2bd7e00700 1 Processor -- start 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.217+0000 7f2bd7e00700 1 -- start start 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 0x7f2bd819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd810d490 0x7f2bd81a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a3540 con 0x7f2bd810d490 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a36b0 con 0x7f2bd8108db0 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7e00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a3870 con 0x7f2bd8108400 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 0x7f2bd819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd810d490 0x7f2bd81a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:41136/0 (socket says 172.21.15.107:41136) 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 -- 172.21.15.107:0/2970427923 learned_addr learned my addr 172.21.15.107:0/2970427923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 -- 172.21.15.107:0/2970427923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 msgr2=0x7f2bd819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 0x7f2bd819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 -- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd810d490 msgr2=0x7f2bd81a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd810d490 0x7f2bd81a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6a00700 1 -- 172.21.15.107:0/2970427923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40095f0 con 0x7f2bd8108400 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.218+0000 7f2bd6000700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 0x7f2bd819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bd6a00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9a0 tx=0x7f2bc800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8009790 con 0x7f2bd8108400 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bc8010430 con 0x7f2bd8108400 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd81a3b50 con 0x7f2bd8108400 2024-03-11T19:51:34.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8018680 con 0x7f2bd8108400 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.219+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd81ab3b0 con 0x7f2bd8108400 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.220+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd81107b0 con 0x7f2bd8108400 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.222+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2bc8010910 con 0x7f2bd8108400 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.223+0000 7f2bcf400700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 0x7f2bbc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.223+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2bc8014070 con 0x7f2bd8108400 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.224+0000 7f2bd6000700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 0x7f2bbc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.224+0000 7f2bd6000700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 0x7f2bbc0a4010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4006010 tx=0x7f2bc4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.228+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2bc8052d20 con 0x7f2bd8108400 2024-03-11T19:51:34.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.463+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2bd819f9c0 con 0x7f2bd8108400 2024-03-11T19:51:34.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.464+0000 7f2bcf400700 1 -- 172.21.15.107:0/2970427923 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bc8096870 con 0x7f2bd8108400 2024-03-11T19:51:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:141733920842 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 msgr2=0x7f2bbc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 0x7f2bbc0a4010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4006010 tx=0x7f2bc4005960 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 msgr2=0x7f2bd819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9a0 tx=0x7f2bc800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 shutdown_connections 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2bbc0a1b50 0x7f2bbc0a4010 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd810d490 0x7f2bd81a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2bd8108400 0x7f2bd819e6b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 --2- 172.21.15.107:0/2970427923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bd8108db0 0x7f2bd819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:34.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 >> 172.21.15.107:0/2970427923 conn(0x7f2bd8077910 msgr2=0x7f2bd8079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:34.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 shutdown_connections 2024-03-11T19:51:34.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.467+0000 7f2bd7e00700 1 -- 172.21.15.107:0/2970427923 wait complete. 2024-03-11T19:51:34.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:34.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:34.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:34.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:34.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:34.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:51:35.273 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:35 smithi107 ceph-mon[31712]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:35.273 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:35 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2970427923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:51:35.273 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:35 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4279806556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:51:35.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:35 smithi122 ceph-mon[36397]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:35.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:35 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2970427923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:51:35.513 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:35 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4279806556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:51:35.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:35 smithi145 ceph-mon[36730]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:35.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:35 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2970427923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:51:35.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:35 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4279806556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:51:36.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 -- 172.21.15.107:0/2964111773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba8112e40 msgr2=0x7f4ba8115230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 --2- 172.21.15.107:0/2964111773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba8112e40 0x7f4ba8115230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009f90 tx=0x7f4ba000b330 comp rx=0 tx=0).stop 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 -- 172.21.15.107:0/2964111773 shutdown_connections 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 --2- 172.21.15.107:0/2964111773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba8112e40 0x7f4ba8115230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 --2- 172.21.15.107:0/2964111773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8073ea0 0x7f4ba8074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 --2- 172.21.15.107:0/2964111773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80747c0 0x7f4ba8112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 -- 172.21.15.107:0/2964111773 >> 172.21.15.107:0/2964111773 conn(0x7f4ba8078dd0 msgr2=0x7f4ba80791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 -- 172.21.15.107:0/2964111773 shutdown_connections 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.832+0000 7f4bae800700 1 -- 172.21.15.107:0/2964111773 wait complete. 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.833+0000 7f4bae800700 1 Processor -- start 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 -- start start 2024-03-11T19:51:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 0x7f4ba81a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 0x7f4ba81a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8112e40 0x7f4ba819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819ef10 con 0x7f4ba8073ea0 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819f080 con 0x7f4ba80747c0 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4bae800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba819f240 con 0x7f4ba8112e40 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4ba7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 0x7f4ba81a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4b9fe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 0x7f4ba81a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4ba7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 0x7f4ba81a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53182/0 (socket says 172.21.15.107:53182) 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4ba7400700 1 -- 172.21.15.107:0/4279806556 learned_addr learned my addr 172.21.15.107:0/4279806556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.834+0000 7f4ba7e00700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8112e40 0x7f4ba819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 -- 172.21.15.107:0/4279806556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8112e40 msgr2=0x7f4ba819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8112e40 0x7f4ba819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 -- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 msgr2=0x7f4ba81a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 0x7f4ba81a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 -- 172.21.15.107:0/4279806556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900095f0 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4b9fe00700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 0x7f4ba81a4b80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005fd0 tx=0x7f4b9000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cbd0 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b9000cd30 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba0009c40 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b900187b0 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.835+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba81ab5c0 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.836+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80755d0 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.838+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f4b90010a30 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.840+0000 7f4ba4c00700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 0x7f4b8c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.840+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4b90014070 con 0x7f4ba80747c0 2024-03-11T19:51:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.840+0000 7f4ba7400700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 0x7f4b8c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:36.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.841+0000 7f4ba7400700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 0x7f4b8c0a4090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009d20 tx=0x7f4b98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:36.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:34.842+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b90097060 con 0x7f4ba80747c0 2024-03-11T19:51:36.537 INFO:teuthology.orchestra.run.smithi107.stdout:107374182478 2024-03-11T19:51:36.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.066+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ba819fc40 con 0x7f4ba80747c0 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.067+0000 7f4ba4c00700 1 -- 172.21.15.107:0/4279806556 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b9009a720 con 0x7f4ba80747c0 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 msgr2=0x7f4b8c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 0x7f4b8c0a4090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009d20 tx=0x7f4b98009260 comp rx=0 tx=0).stop 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 msgr2=0x7f4ba81a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 0x7f4ba81a4b80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005fd0 tx=0x7f4b9000ee70 comp rx=0 tx=0).stop 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 shutdown_connections 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b8c0a1bd0 0x7f4b8c0a4090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4ba8112e40 0x7f4ba819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba8073ea0 0x7f4ba81a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 --2- 172.21.15.107:0/4279806556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ba80747c0 0x7f4ba81a4b80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 >> 172.21.15.107:0/4279806556 conn(0x7f4ba8078dd0 msgr2=0x7f4ba8111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.070+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 shutdown_connections 2024-03-11T19:51:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:35.071+0000 7f4bae800700 1 -- 172.21.15.107:0/4279806556 wait complete. 2024-03-11T19:51:37.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:37 smithi107 ceph-mon[31712]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:37.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:37 smithi122 ceph-mon[36397]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:37.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:37 smithi145 ceph-mon[36730]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 -- 172.21.15.107:0/2513991313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4102760 msgr2=0x7f73b41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 --2- 172.21.15.107:0/2513991313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4102760 0x7f73b41067b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009990 tx=0x7f73a0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 -- 172.21.15.107:0/2513991313 shutdown_connections 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 --2- 172.21.15.107:0/2513991313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4106e90 0x7f73b410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 --2- 172.21.15.107:0/2513991313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 --2- 172.21.15.107:0/2513991313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4102760 0x7f73b41067b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 -- 172.21.15.107:0/2513991313 >> 172.21.15.107:0/2513991313 conn(0x7f73b40fd660 msgr2=0x7f73b40ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 -- 172.21.15.107:0/2513991313 shutdown_connections 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.683+0000 7f73ba600700 1 -- 172.21.15.107:0/2513991313 wait complete. 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.684+0000 7f73ba600700 1 Processor -- start 2024-03-11T19:51:37.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.684+0000 7f73ba600700 1 -- start start 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4102760 0x7f73b40744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4106e90 0x7f73b40749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41a6560 con 0x7f73b4101db0 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41a66d0 con 0x7f73b4102760 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73ba600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41a6890 con 0x7f73b4106e90 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73b9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73b3e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4102760 0x7f73b40744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73b9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46332/0 (socket says 172.21.15.107:46332) 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73b9200700 1 -- 172.21.15.107:0/810738748 learned_addr learned my addr 172.21.15.107:0/810738748 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.685+0000 7f73b9c00700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4106e90 0x7f73b40749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 -- 172.21.15.107:0/810738748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4106e90 msgr2=0x7f73b40749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4106e90 0x7f73b40749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 -- 172.21.15.107:0/810738748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4102760 msgr2=0x7f73b40744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4102760 0x7f73b40744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 -- 172.21.15.107:0/810738748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a00095f0 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b9200700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e9a0 tx=0x7f73a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8009790 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b41a6b70 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73a8010430 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8018680 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.686+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b41a7090 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.688+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b410a150 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.690+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f73a8010910 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.692+0000 7f73b1600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 0x7f739c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.693+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f73a8014070 con 0x7f73b4101db0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.693+0000 7f73b3e00700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 0x7f739c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.693+0000 7f73b3e00700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 0x7f739c0a4010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73b4075a00 tx=0x7f73a000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:37.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.697+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73a8052d20 con 0x7f73b4101db0 2024-03-11T19:51:37.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.921+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73b40754a0 con 0x7f73b4101db0 2024-03-11T19:51:37.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.922+0000 7f73b1600700 1 -- 172.21.15.107:0/810738748 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73a8096850 con 0x7f73b4101db0 2024-03-11T19:51:37.966 INFO:teuthology.orchestra.run.smithi107.stdout:55834574939 2024-03-11T19:51:37.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 msgr2=0x7f739c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:37.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 0x7f739c0a4010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73b4075a00 tx=0x7f73a000b260 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 msgr2=0x7f73b4073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e9a0 tx=0x7f73a800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 shutdown_connections 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f739c0a1b50 0x7f739c0a4010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73b4106e90 0x7f73b40749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73b4101db0 0x7f73b4073f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 --2- 172.21.15.107:0/810738748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73b4102760 0x7f73b40744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 >> 172.21.15.107:0/810738748 conn(0x7f73b40fd660 msgr2=0x7f73b40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 shutdown_connections 2024-03-11T19:51:37.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:37.925+0000 7f73ba600700 1 -- 172.21.15.107:0/810738748 wait complete. 2024-03-11T19:51:38.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:38 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/810738748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:51:38.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:38 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/810738748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:51:38.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:38 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/810738748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:51:39.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 -- 172.21.15.107:0/3654404411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c10d490 msgr2=0x7f634c115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 --2- 172.21.15.107:0/3654404411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c10d490 0x7f634c115060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6344009990 tx=0x7f6344009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 -- 172.21.15.107:0/3654404411 shutdown_connections 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 --2- 172.21.15.107:0/3654404411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c10d490 0x7f634c115060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 --2- 172.21.15.107:0/3654404411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 --2- 172.21.15.107:0/3654404411 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c108db0 0x7f634c10cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 -- 172.21.15.107:0/3654404411 >> 172.21.15.107:0/3654404411 conn(0x7f634c077910 msgr2=0x7f634c079d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 -- 172.21.15.107:0/3654404411 shutdown_connections 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.487+0000 7f6352e00700 1 -- 172.21.15.107:0/3654404411 wait complete. 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 Processor -- start 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 -- start start 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 0x7f634c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c10d490 0x7f634c1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c1a34c0 con 0x7f634c108db0 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c1a3630 con 0x7f634c108400 2024-03-11T19:51:39.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.489+0000 7f6352e00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c1a37f0 con 0x7f634c10d490 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 0x7f634c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:54568/0 (socket says 172.21.15.107:54568) 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 -- 172.21.15.107:0/952376418 learned_addr learned my addr 172.21.15.107:0/952376418 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6352400700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c10d490 0x7f634c1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 -- 172.21.15.107:0/952376418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c10d490 msgr2=0x7f634c1a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c10d490 0x7f634c1a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 -- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 msgr2=0x7f634c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 0x7f634c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351a00700 1 -- 172.21.15.107:0/952376418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63440095f0 con 0x7f634c108400 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.490+0000 7f6351000700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 0x7f634c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f6351a00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f634000ef10 tx=0x7f634000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000c9c0 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f634000f040 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340014480 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c1a3a70 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.491+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c1ab330 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.492+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c1107b0 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.494+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6340007720 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.496+0000 7f634a000700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 0x7f63340a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.496+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f63400cdb90 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.496+0000 7f6351000700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 0x7f63340a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.497+0000 7f6351000700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 0x7f63340a4090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f634c107560 tx=0x7f6338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.500+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6340096880 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.725+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f634c19f940 con 0x7f634c108400 2024-03-11T19:51:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.726+0000 7f634a000700 1 -- 172.21.15.107:0/952376418 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6340099f40 con 0x7f634c108400 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stdout:219043332152 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 msgr2=0x7f63340a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 0x7f63340a4090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f634c107560 tx=0x7f6338009260 comp rx=0 tx=0).stop 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 msgr2=0x7f634c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f634000ef10 tx=0x7f634000c3a0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 shutdown_connections 2024-03-11T19:51:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f63340a1bd0 0x7f63340a4090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f634c10d490 0x7f634c1a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f634c108400 0x7f634c19e5e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 --2- 172.21.15.107:0/952376418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c108db0 0x7f634c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.730+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 >> 172.21.15.107:0/952376418 conn(0x7f634c077910 msgr2=0x7f634c079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.731+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 shutdown_connections 2024-03-11T19:51:39.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:38.731+0000 7f6352e00700 1 -- 172.21.15.107:0/952376418 wait complete. 2024-03-11T19:51:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:39 smithi122 ceph-mon[36397]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:39 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/952376418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:51:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:39 smithi145 ceph-mon[36730]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:39 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/952376418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:51:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:39 smithi107 ceph-mon[31712]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:39 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/952376418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:51:40.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920842 for osd.4 2024-03-11T19:51:40.065 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 -- 172.21.15.107:0/2634977743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 msgr2=0x7fa0bc074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/2634977743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc074280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009990 tx=0x7fa0ac009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 -- 172.21.15.107:0/2634977743 shutdown_connections 2024-03-11T19:51:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/2634977743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 0x7fa0bc110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/2634977743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc074280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/2634977743 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 -- 172.21.15.107:0/2634977743 >> 172.21.15.107:0/2634977743 conn(0x7fa0bc078dd0 msgr2=0x7fa0bc0791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 -- 172.21.15.107:0/2634977743 shutdown_connections 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.522+0000 7fa0c0a00700 1 -- 172.21.15.107:0/2634977743 wait complete. 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 Processor -- start 2024-03-11T19:51:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 -- start start 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 0x7fa0bc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19f320 con 0x7fa0bc073ea0 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19f490 con 0x7fa0bc10e670 2024-03-11T19:51:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.524+0000 7fa0c0a00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19f650 con 0x7fa0bc0747c0 2024-03-11T19:51:41.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:58028/0 (socket says 172.21.15.107:58028) 2024-03-11T19:51:41.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 -- 172.21.15.107:0/4219903759 learned_addr learned my addr 172.21.15.107:0/4219903759 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0ba800700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 0x7fa0bc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 -- 172.21.15.107:0/4219903759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 msgr2=0x7fa0bc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 0x7fa0bc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 -- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 msgr2=0x7fa0bc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 -- 172.21.15.107:0/4219903759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4009c40 con 0x7fa0bc0747c0 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.525+0000 7fa0b9400700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9a0 tx=0x7fa0a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.526+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8009790 con 0x7fa0bc0747c0 2024-03-11T19:51:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.526+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0095f0 con 0x7fa0bc0747c0 2024-03-11T19:51:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.526+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8010430 con 0x7fa0bc0747c0 2024-03-11T19:51:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.526+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8018680 con 0x7fa0bc0747c0 2024-03-11T19:51:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.526+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc1a6fc0 con 0x7fa0bc0747c0 2024-03-11T19:51:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.527+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc109880 con 0x7fa0bc0747c0 2024-03-11T19:51:41.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.529+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fa0a8010910 con 0x7fa0bc0747c0 2024-03-11T19:51:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.530+0000 7fa0b2a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 0x7fa0a00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.530+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa0a8014070 con 0x7fa0bc0747c0 2024-03-11T19:51:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.531+0000 7fa0b9e00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 0x7fa0a00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.531+0000 7fa0b9e00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 0x7fa0a00a4010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.534+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0a8052d20 con 0x7fa0bc0747c0 2024-03-11T19:51:41.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:41 smithi107 ceph-mon[31712]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:41.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:41 smithi122 ceph-mon[36397]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:41.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.770+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa0bc068930 con 0x7fa0bc0747c0 2024-03-11T19:51:41.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.770+0000 7fa0b2a00700 1 -- 172.21.15.107:0/4219903759 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0a8096870 con 0x7fa0bc0747c0 2024-03-11T19:51:41.775 INFO:teuthology.orchestra.run.smithi107.stdout:34359738465 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 msgr2=0x7fa0a00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 0x7fa0a00a4010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006010 tx=0x7fa0ac00b260 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 msgr2=0x7fa0bc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e9a0 tx=0x7fa0a800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 shutdown_connections 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa0a00a1b50 0x7fa0a00a4010 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0bc10e670 0x7fa0bc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0bc073ea0 0x7fa0bc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 --2- 172.21.15.107:0/4219903759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa0bc0747c0 0x7fa0bc19a9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.773+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 >> 172.21.15.107:0/4219903759 conn(0x7fa0bc078dd0 msgr2=0x7fa0bc0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.774+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 shutdown_connections 2024-03-11T19:51:41.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.774+0000 7fa0c0a00700 1 -- 172.21.15.107:0/4219903759 wait complete. 2024-03-11T19:51:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:41 smithi145 ceph-mon[36730]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.885+0000 7f98afe00700 1 -- 172.21.15.107:0/833748078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8112b20 msgr2=0x7f98a8114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.885+0000 7f98afe00700 1 --2- 172.21.15.107:0/833748078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8112b20 0x7f98a8114f10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98a000b0a0 tx=0x7f98a000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 -- 172.21.15.107:0/833748078 shutdown_connections 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 --2- 172.21.15.107:0/833748078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8112b20 0x7f98a8114f10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 --2- 172.21.15.107:0/833748078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a80761a0 0x7f98a8076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 --2- 172.21.15.107:0/833748078 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a8076ac0 0x7f98a81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 -- 172.21.15.107:0/833748078 >> 172.21.15.107:0/833748078 conn(0x7f98a80ff970 msgr2=0x7f98a8101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 -- 172.21.15.107:0/833748078 shutdown_connections 2024-03-11T19:51:41.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.886+0000 7f98afe00700 1 -- 172.21.15.107:0/833748078 wait complete. 2024-03-11T19:51:41.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.887+0000 7f98afe00700 1 Processor -- start 2024-03-11T19:51:41.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.887+0000 7f98afe00700 1 -- start start 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a80761a0 0x7f98a81a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a8112b20 0x7f98a819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a819eee0 con 0x7f98a8076ac0 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a819f050 con 0x7f98a8112b20 2024-03-11T19:51:41.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98afe00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a819f210 con 0x7f98a80761a0 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98aca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98aca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46394/0 (socket says 172.21.15.107:46394) 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98aca00700 1 -- 172.21.15.107:0/705777124 learned_addr learned my addr 172.21.15.107:0/705777124 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.888+0000 7f98ade00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a8112b20 0x7f98a819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98ad400700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a80761a0 0x7f98a81a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 -- 172.21.15.107:0/705777124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a80761a0 msgr2=0x7f98a81a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a80761a0 0x7f98a81a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 -- 172.21.15.107:0/705777124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a8112b20 msgr2=0x7f98a819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a8112b20 0x7f98a819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 -- 172.21.15.107:0/705777124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009520 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98aca00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9a0 tx=0x7f989400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894009790 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f989400cbd0 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894018680 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0009d00 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.889+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a81ab5e0 con 0x7f98a8076ac0 2024-03-11T19:51:41.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.891+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a80778e0 con 0x7f98a8076ac0 2024-03-11T19:51:41.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.893+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f989400cd40 con 0x7f98a8076ac0 2024-03-11T19:51:41.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.894+0000 7f98a6000700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 0x7f98900a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.894+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9894014070 con 0x7f98a8076ac0 2024-03-11T19:51:41.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.894+0000 7f98ad400700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 0x7f98900a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.895+0000 7f98ad400700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 0x7f98900a4090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d80 tx=0x7f989c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:41.898+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9894052d20 con 0x7f98a8076ac0 2024-03-11T19:51:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.122+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98a8068930 con 0x7f98a8076ac0 2024-03-11T19:51:42.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.123+0000 7f98a6000700 1 -- 172.21.15.107:0/705777124 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9894096820 con 0x7f98a8076ac0 2024-03-11T19:51:42.127 INFO:teuthology.orchestra.run.smithi107.stdout:77309411415 2024-03-11T19:51:42.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.125+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 msgr2=0x7f98900a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 0x7f98900a4090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f989c009d80 tx=0x7f989c009260 comp rx=0 tx=0).stop 2024-03-11T19:51:42.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 msgr2=0x7f98a81a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9a0 tx=0x7f989400ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 shutdown_connections 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98900a1bd0 0x7f98900a4090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a8112b20 0x7f98a819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a80761a0 0x7f98a81a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 --2- 172.21.15.107:0/705777124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a8076ac0 0x7f98a81a4ba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 >> 172.21.15.107:0/705777124 conn(0x7f98a80ff970 msgr2=0x7f98a8101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 shutdown_connections 2024-03-11T19:51:42.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:42.126+0000 7f98afe00700 1 -- 172.21.15.107:0/705777124 wait complete. 2024-03-11T19:51:42.685 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:42 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4219903759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:51:42.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:42 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/705777124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:51:42.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:42 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4219903759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:51:42.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:42 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/705777124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:51:42.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:42 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4219903759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:51:42.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:42 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/705777124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:51:43.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332145 got 219043332152 for osd.7 2024-03-11T19:51:43.274 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:43.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182478 for osd.3 2024-03-11T19:51:43.626 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:43.707 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:43 smithi107 ceph-mon[31712]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:43.747 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:43.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:43 smithi122 ceph-mon[36397]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:43 smithi145 ceph-mon[36730]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.707+0000 7f6e4ca00700 1 -- 172.21.15.107:0/693433873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 msgr2=0x7f6e48108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.707+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/693433873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e48108aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009990 tx=0x7f6e34009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 -- 172.21.15.107:0/693433873 shutdown_connections 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/693433873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e48109180 0x7f6e48110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/693433873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e481040a0 0x7f6e48104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/693433873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e48108aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 -- 172.21.15.107:0/693433873 >> 172.21.15.107:0/693433873 conn(0x7f6e480ff950 msgr2=0x7f6e48101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:44.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 -- 172.21.15.107:0/693433873 shutdown_connections 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.708+0000 7f6e4ca00700 1 -- 172.21.15.107:0/693433873 wait complete. 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 Processor -- start 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 -- start start 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e4819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 0x7f6e4819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4819f2d0 con 0x7f6e481040a0 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4819f440 con 0x7f6e48104a50 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.709+0000 7f6e4ca00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4819f600 con 0x7f6e48109180 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e4819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 0x7f6e4819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46408/0 (socket says 172.21.15.107:46408) 2024-03-11T19:51:44.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45e00700 1 -- 172.21.15.107:0/1015891910 learned_addr learned my addr 172.21.15.107:0/1015891910 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 -- 172.21.15.107:0/1015891910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 msgr2=0x7f6e4819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e4819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 -- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 msgr2=0x7f6e4819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e46800700 1 -- 172.21.15.107:0/1015891910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340095f0 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45400700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e4819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.710+0000 7f6e45e00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e46800700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 0x7f6e4819ecc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000efc0 tx=0x7f6e4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40018070 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e4000f040 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4819f880 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000cd10 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.711+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e481a7140 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.713+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4810c4a0 con 0x7f6e48109180 2024-03-11T19:51:44.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.714+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f6e40004ad0 con 0x7f6e48109180 2024-03-11T19:51:44.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.715+0000 7f6e3ea00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 0x7f6e2c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:44.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.716+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6e400cd2a0 con 0x7f6e48109180 2024-03-11T19:51:44.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.716+0000 7f6e45e00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 0x7f6e2c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:44.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.716+0000 7f6e45e00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 0x7f6e2c0a4090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6e38011550 tx=0x7f6e38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:44.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.720+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e40096080 con 0x7f6e48109180 2024-03-11T19:51:44.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.949+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6e4819b750 con 0x7f6e48109180 2024-03-11T19:51:44.955 INFO:teuthology.orchestra.run.smithi107.stdout:197568495679 2024-03-11T19:51:44.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.949+0000 7f6e3ea00700 1 -- 172.21.15.107:0/1015891910 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e40099740 con 0x7f6e48109180 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 msgr2=0x7f6e2c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 0x7f6e2c0a4090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6e38011550 tx=0x7f6e38009260 comp rx=0 tx=0).stop 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 msgr2=0x7f6e4819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 0x7f6e4819ecc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000efc0 tx=0x7f6e4000c3a0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 shutdown_connections 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e2c0a1bd0 0x7f6e2c0a4090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e48109180 0x7f6e4819ecc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e481040a0 0x7f6e4819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 --2- 172.21.15.107:0/1015891910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e48104a50 0x7f6e4819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:44.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 >> 172.21.15.107:0/1015891910 conn(0x7f6e480ff950 msgr2=0x7f6e48101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:44.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 shutdown_connections 2024-03-11T19:51:44.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:44.953+0000 7f6e4ca00700 1 -- 172.21.15.107:0/1015891910 wait complete. 2024-03-11T19:51:45.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:45 smithi107 ceph-mon[31712]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:45.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:45 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1015891910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:51:45.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:45 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2111790518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:51:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:45 smithi122 ceph-mon[36397]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:45 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1015891910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:51:45.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:45 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2111790518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:51:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:45 smithi145 ceph-mon[36730]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:45 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1015891910' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:51:45.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:45 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2111790518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:51:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 -- 172.21.15.107:0/3072254528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 msgr2=0x7f2a7c1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 --2- 172.21.15.107:0/3072254528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1096c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009990 tx=0x7f2a68009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 -- 172.21.15.107:0/3072254528 shutdown_connections 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 --2- 172.21.15.107:0/3072254528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c119190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 --2- 172.21.15.107:0/3072254528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 0x7f2a7c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 --2- 172.21.15.107:0/3072254528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1096c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 -- 172.21.15.107:0/3072254528 >> 172.21.15.107:0/3072254528 conn(0x7f2a7c06d190 msgr2=0x7f2a7c06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 -- 172.21.15.107:0/3072254528 shutdown_connections 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.031+0000 7f2a81c00700 1 -- 172.21.15.107:0/3072254528 wait complete. 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.032+0000 7f2a81c00700 1 Processor -- start 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.033+0000 7f2a81c00700 1 -- start start 2024-03-11T19:51:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.033+0000 7f2a81c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.033+0000 7f2a81c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 0x7f2a7c1a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.033+0000 7f2a81c00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c110940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.033+0000 7f2a81c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c110f10 con 0x7f2a7c10c420 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a81c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c111050 con 0x7f2a7c109240 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a81c00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c111210 con 0x7f2a7c116da0 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 0x7f2a7c1a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:53496/0 (socket says 172.21.15.107:53496) 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 -- 172.21.15.107:0/2111790518 learned_addr learned my addr 172.21.15.107:0/2111790518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7b400700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c110940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 -- 172.21.15.107:0/2111790518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 msgr2=0x7f2a7c110940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c110940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 -- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 msgr2=0x7f2a7c1a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 0x7f2a7c1a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 -- 172.21.15.107:0/2111790518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a680095f0 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7b400700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.034+0000 7f2a7aa00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9a0 tx=0x7f2a6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.035+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c009790 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.035+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c010430 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.035+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c018680 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.035+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1114f0 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.035+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c115bf0 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.036+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c068930 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.039+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f2a6c010910 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.040+0000 7f2a73400700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 0x7f2a600a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.041+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2a6c014070 con 0x7f2a7c109240 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.041+0000 7f2a7a000700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 0x7f2a600a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.042+0000 7f2a7a000700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 0x7f2a600a4090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b2e0 tx=0x7f2a68005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.045+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a6c052d20 con 0x7f2a7c109240 2024-03-11T19:51:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.270+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2a7c11c6a0 con 0x7f2a7c109240 2024-03-11T19:51:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.271+0000 7f2a73400700 1 -- 172.21.15.107:0/2111790518 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a6c0967d0 con 0x7f2a7c109240 2024-03-11T19:51:46.021 INFO:teuthology.orchestra.run.smithi107.stdout:287762808879 2024-03-11T19:51:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 msgr2=0x7f2a600a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 0x7f2a600a4090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b2e0 tx=0x7f2a68005960 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 msgr2=0x7f2a7c1a8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9a0 tx=0x7f2a6c00ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 shutdown_connections 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2a600a1bd0 0x7f2a600a4090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a7c116da0 0x7f2a7c110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7c109240 0x7f2a7c1a8b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 --2- 172.21.15.107:0/2111790518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a7c10c420 0x7f2a7c1a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.274+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 >> 172.21.15.107:0/2111790518 conn(0x7f2a7c06d190 msgr2=0x7f2a7c118490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.275+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 shutdown_connections 2024-03-11T19:51:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:45.275+0000 7f2a81c00700 1 -- 172.21.15.107:0/2111790518 wait complete. 2024-03-11T19:51:46.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574939 for osd.1 2024-03-11T19:51:46.206 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:46.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738458 got 34359738465 for osd.0 2024-03-11T19:51:46.877 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:47.498 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:47 smithi107 ceph-mon[31712]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:47.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:47 smithi122 ceph-mon[36397]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:47.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:47 smithi145 ceph-mon[36730]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:51:48.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 -- 172.21.15.107:0/1648443597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c1036f0 msgr2=0x7fae0c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:48.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 --2- 172.21.15.107:0/1648443597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c1036f0 0x7fae0c10bbe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadf40098f0 tx=0x7fadf4009c00 comp rx=0 tx=0).stop 2024-03-11T19:51:48.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 -- 172.21.15.107:0/1648443597 shutdown_connections 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 --2- 172.21.15.107:0/1648443597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c10c1b0 0x7fae0c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 --2- 172.21.15.107:0/1648443597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 0x7fae0c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 --2- 172.21.15.107:0/1648443597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c1036f0 0x7fae0c10bbe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 -- 172.21.15.107:0/1648443597 >> 172.21.15.107:0/1648443597 conn(0x7fae0c0fc540 msgr2=0x7fae0c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 -- 172.21.15.107:0/1648443597 shutdown_connections 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.091+0000 7fae11a00700 1 -- 172.21.15.107:0/1648443597 wait complete. 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.092+0000 7fae11a00700 1 Processor -- start 2024-03-11T19:51:48.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.092+0000 7fae11a00700 1 -- start start 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 0x7fae0c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c10c1b0 0x7fae0c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f1e0 con 0x7fae0c1036f0 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f350 con 0x7fae0c102dd0 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11a00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f510 con 0x7fae0c10c1b0 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:48.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35610/0 (socket says 172.21.15.107:35610) 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 -- 172.21.15.107:0/1402414197 learned_addr learned my addr 172.21.15.107:0/1402414197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae11000700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c10c1b0 0x7fae0c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae0be00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 0x7fae0c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 -- 172.21.15.107:0/1402414197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c10c1b0 msgr2=0x7fae0c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c10c1b0 0x7fae0c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 -- 172.21.15.107:0/1402414197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 msgr2=0x7fae0c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 0x7fae0c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.093+0000 7fae03e00700 1 -- 172.21.15.107:0/1402414197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4009520 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae03e00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fadf400bf50 tx=0x7fadf400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf401c070 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadf400b560 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c19f790 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4003c40 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.094+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c1a7050 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.095+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c068930 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.097+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7fadf400b6d0 con 0x7fae0c1036f0 2024-03-11T19:51:48.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.098+0000 7fae09600700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 0x7fadf00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.099+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fadf40ce880 con 0x7fae0c1036f0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.099+0000 7fae0be00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 0x7fadf00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.100+0000 7fae0be00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 0x7fadf00a4010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009c60 tx=0x7fadfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.102+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadf4097600 con 0x7fae0c1036f0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.328+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fae0c111be0 con 0x7fae0c1036f0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.328+0000 7fae09600700 1 -- 172.21.15.107:0/1402414197 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadf409acc0 con 0x7fae0c1036f0 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 msgr2=0x7fadf00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:48.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 0x7fadf00a4010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009c60 tx=0x7fadfc009260 comp rx=0 tx=0).stop 2024-03-11T19:51:48.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 msgr2=0x7fae0c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:48.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fadf400bf50 tx=0x7fadf400fa70 comp rx=0 tx=0).stop 2024-03-11T19:51:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:48 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1402414197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stdout:253403070518 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 shutdown_connections 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadf00a1b50 0x7fadf00a4010 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae0c10c1b0 0x7fae0c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae0c102dd0 0x7fae0c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 --2- 172.21.15.107:0/1402414197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae0c1036f0 0x7fae0c19a840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.331+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 >> 172.21.15.107:0/1402414197 conn(0x7fae0c0fc540 msgr2=0x7fae0c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.332+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 shutdown_connections 2024-03-11T19:51:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:48.332+0000 7fae11a00700 1 -- 172.21.15.107:0/1402414197 wait complete. 2024-03-11T19:51:48.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:48 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1402414197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:51:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:48 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1402414197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:51:49.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 -- 172.21.15.107:0/3514190650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 msgr2=0x7f13c81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 --2- 172.21.15.107:0/3514190650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c81021a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009990 tx=0x7f13bc009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 -- 172.21.15.107:0/3514190650 shutdown_connections 2024-03-11T19:51:49.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 --2- 172.21.15.107:0/3514190650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8106ea0 0x7f13c810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 --2- 172.21.15.107:0/3514190650 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c81021a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 --2- 172.21.15.107:0/3514190650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8102770 0x7f13c81067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 -- 172.21.15.107:0/3514190650 >> 172.21.15.107:0/3514190650 conn(0x7f13c80fd670 msgr2=0x7f13c80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:49.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 -- 172.21.15.107:0/3514190650 shutdown_connections 2024-03-11T19:51:49.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:49 smithi107 ceph-mon[31712]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:49.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.501+0000 7f13cea00700 1 -- 172.21.15.107:0/3514190650 wait complete. 2024-03-11T19:51:49.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 Processor -- start 2024-03-11T19:51:49.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 -- start start 2024-03-11T19:51:49.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 0x7f13c819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 0x7f13c819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c819f230 con 0x7f13c8102770 2024-03-11T19:51:49.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.503+0000 7f13cea00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c819f3a0 con 0x7f13c8106ea0 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13cea00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c819f560 con 0x7f13c8101dc0 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13cd600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 0x7f13c819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13cd600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40122/0 (socket says 172.21.15.107:40122) 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13cd600700 1 -- 172.21.15.107:0/2225576251 learned_addr learned my addr 172.21.15.107:0/2225576251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ce000700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 0x7f13c819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 -- 172.21.15.107:0/2225576251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 msgr2=0x7f13c819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 -- 172.21.15.107:0/2225576251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 msgr2=0x7f13c819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 0x7f13c819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ccc00700 1 -- 172.21.15.107:0/2225576251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0009c40 con 0x7f13c8102770 2024-03-11T19:51:49.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13cd600700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:49.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.504+0000 7f13ce000700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 0x7f13c819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:49.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13ccc00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 0x7f13c819a890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e9c0 tx=0x7f13b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:49.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b400c9c0 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b4010430 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc0095f0 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4018680 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.505+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c81a6ed0 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.506+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c8068930 con 0x7f13c8102770 2024-03-11T19:51:49.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.508+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f13b4010910 con 0x7f13c8102770 2024-03-11T19:51:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.509+0000 7f13c6000700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 0x7f13b00a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.510+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f13b4014070 con 0x7f13c8102770 2024-03-11T19:51:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.510+0000 7f13cd600700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 0x7f13b00a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.510+0000 7f13cd600700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 0x7f13b00a4090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006010 tx=0x7f13bc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:49.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.514+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13b4052d20 con 0x7f13c8102770 2024-03-11T19:51:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.734+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f13c8111c40 con 0x7f13c8102770 2024-03-11T19:51:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.735+0000 7f13c6000700 1 -- 172.21.15.107:0/2225576251 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f13b40967b0 con 0x7f13c8102770 2024-03-11T19:51:49.739 INFO:teuthology.orchestra.run.smithi107.stdout:309237645354 2024-03-11T19:51:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 msgr2=0x7f13b00a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 0x7f13b00a4090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13bc006010 tx=0x7f13bc00b260 comp rx=0 tx=0).stop 2024-03-11T19:51:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 msgr2=0x7f13c819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 0x7f13c819a890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e9c0 tx=0x7f13b400ecd0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 shutdown_connections 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f13b00a1bd0 0x7f13b00a4090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c8106ea0 0x7f13c819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f13c8101dc0 0x7f13c819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 --2- 172.21.15.107:0/2225576251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c8102770 0x7f13c819a890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.739+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 >> 172.21.15.107:0/2225576251 conn(0x7f13c80fd670 msgr2=0x7f13c810dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.740+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 shutdown_connections 2024-03-11T19:51:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.740+0000 7f13cea00700 1 -- 172.21.15.107:0/2225576251 wait complete. 2024-03-11T19:51:49.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:49 smithi122 ceph-mon[36397]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 -- 172.21.15.107:0/579802007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a41084e0 msgr2=0x7f19a41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 --2- 172.21.15.107:0/579802007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a41084e0 0x7f19a41088c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19980098f0 tx=0x7f1998009c00 comp rx=0 tx=0).stop 2024-03-11T19:51:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 -- 172.21.15.107:0/579802007 shutdown_connections 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 --2- 172.21.15.107:0/579802007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a4115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 --2- 172.21.15.107:0/579802007 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a41084e0 0x7f19a41088c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 --2- 172.21.15.107:0/579802007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a4108e90 0x7f19a410cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.772+0000 7f19ac600700 1 -- 172.21.15.107:0/579802007 >> 172.21.15.107:0/579802007 conn(0x7f19a4075440 msgr2=0x7f19a4077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.773+0000 7f19ac600700 1 -- 172.21.15.107:0/579802007 shutdown_connections 2024-03-11T19:51:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.773+0000 7f19ac600700 1 -- 172.21.15.107:0/579802007 wait complete. 2024-03-11T19:51:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.774+0000 7f19ac600700 1 Processor -- start 2024-03-11T19:51:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.774+0000 7f19ac600700 1 -- start start 2024-03-11T19:51:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 0x7f19a41a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 0x7f19a41a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a419ecb0 con 0x7f19a410d5c0 2024-03-11T19:51:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a419edf0 con 0x7f19a41084e0 2024-03-11T19:51:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19ac600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a419efb0 con 0x7f19a4108e90 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19a9200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 0x7f19a41a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19a9c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 0x7f19a41a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19a9200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 0x7f19a41a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40140/0 (socket says 172.21.15.107:40140) 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19a9200700 1 -- 172.21.15.107:0/3238650187 learned_addr learned my addr 172.21.15.107:0/3238650187 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.775+0000 7f19aa600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 -- 172.21.15.107:0/3238650187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 msgr2=0x7f19a41a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 0x7f19a41a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 -- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 msgr2=0x7f19a419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 -- 172.21.15.107:0/3238650187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990009520 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19aa600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a9c00700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 0x7f19a41a4440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1998000c00 tx=0x7f199800fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199801c070 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c009d00 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f199800ba40 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998003d20 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.776+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1998009520 con 0x7f19a41084e0 2024-03-11T19:51:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.778+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a41107f0 con 0x7f19a41084e0 2024-03-11T19:51:49.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.780+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f199800b560 con 0x7f19a41084e0 2024-03-11T19:51:49.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.781+0000 7f19a2000700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 0x7f198c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:49.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.781+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f19980ce310 con 0x7f19a41084e0 2024-03-11T19:51:49.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.782+0000 7f19a9200700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 0x7f198c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:49.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.782+0000 7f19a9200700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 0x7f198c0a4010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19a4076860 tx=0x7f1990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:49.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:49 smithi145 ceph-mon[36730]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:49.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:49.785+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1998097090 con 0x7f19a41084e0 2024-03-11T19:51:49.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808873 got 287762808879 for osd.9 2024-03-11T19:51:49.955 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:50.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.017+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f19a419fb30 con 0x7f19a41084e0 2024-03-11T19:51:50.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.017+0000 7f19a2000700 1 -- 172.21.15.107:0/3238650187 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f199809fe60 con 0x7f19a41084e0 2024-03-11T19:51:50.021 INFO:teuthology.orchestra.run.smithi107.stdout:163208757319 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.020+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 msgr2=0x7f198c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.020+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 0x7f198c0a4010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19a4076860 tx=0x7f1990009260 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 msgr2=0x7f19a41a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 0x7f19a41a4440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1998000c00 tx=0x7f199800fa90 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 shutdown_connections 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f198c0a1b50 0x7f198c0a4010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a410d5c0 0x7f19a419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19a41084e0 0x7f19a41a4440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 --2- 172.21.15.107:0/3238650187 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f19a4108e90 0x7f19a41a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 >> 172.21.15.107:0/3238650187 conn(0x7f19a4075440 msgr2=0x7f19a410c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 shutdown_connections 2024-03-11T19:51:50.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:50.021+0000 7f19ac600700 1 -- 172.21.15.107:0/3238650187 wait complete. 2024-03-11T19:51:50.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:50 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2225576251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:51:50.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:50 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3238650187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:51:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:50 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2225576251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:51:50.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:50 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3238650187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:51:50.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:50 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2225576251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:51:50.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:50 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3238650187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:51:50.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495674 got 197568495679 for osd.6 2024-03-11T19:51:50.937 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:51.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411411 got 77309411415 for osd.2 2024-03-11T19:51:51.533 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:51.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:51 smithi107 ceph-mon[31712]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:51 smithi122 ceph-mon[36397]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:51 smithi145 ceph-mon[36730]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:51:52.148 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757312 got 163208757319 for osd.5 2024-03-11T19:51:52.149 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:52.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070511 got 253403070518 for osd.8 2024-03-11T19:51:52.532 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:53.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645346 got 309237645354 for osd.10 2024-03-11T19:51:53.556 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:53 smithi122 ceph-mon[36397]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:53 smithi145 ceph-mon[36730]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:53 smithi107 ceph-mon[31712]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 -- 172.21.15.107:0/787085644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 msgr2=0x7f64dc10cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 --2- 172.21.15.107:0/787085644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc10cee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009990 tx=0x7f64c4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 -- 172.21.15.107:0/787085644 shutdown_connections 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 --2- 172.21.15.107:0/787085644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc115190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 --2- 172.21.15.107:0/787085644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 0x7f64dc1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 --2- 172.21.15.107:0/787085644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc10cee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 -- 172.21.15.107:0/787085644 >> 172.21.15.107:0/787085644 conn(0x7f64dc075440 msgr2=0x7f64dc077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:54.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 -- 172.21.15.107:0/787085644 shutdown_connections 2024-03-11T19:51:54.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.221+0000 7f64e0a00700 1 -- 172.21.15.107:0/787085644 wait complete. 2024-03-11T19:51:54.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.222+0000 7f64e0a00700 1 Processor -- start 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.222+0000 7f64e0a00700 1 -- start start 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.222+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 0x7f64dc1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.222+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64e0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64e0a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19ec90 con 0x7f64dc10d5c0 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64e0a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19edd0 con 0x7f64dc108e90 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64e0a00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19ef90 con 0x7f64dc1084e0 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64d3e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64d9e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 0x7f64dc1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35666/0 (socket says 172.21.15.107:35666) 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64d3e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:53582/0 (socket says 172.21.15.107:53582) 2024-03-11T19:51:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 -- 172.21.15.107:0/2257500767 learned_addr learned my addr 172.21.15.107:0/2257500767 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:54.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 -- 172.21.15.107:0/2257500767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 msgr2=0x7f64dc1a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:54.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 0x7f64dc1a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 -- 172.21.15.107:0/2257500767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 msgr2=0x7f64dc1a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:54.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64da800700 1 -- 172.21.15.107:0/2257500767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40095f0 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.223+0000 7f64d3e00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64da800700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ef30 tx=0x7f64d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4018070 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc19f210 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64d400f040 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400cd20 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.224+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc1ab4c0 con 0x7f64dc10d5c0 2024-03-11T19:51:54.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.225+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc110830 con 0x7f64dc10d5c0 2024-03-11T19:51:54.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.227+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 136331+0+0 (secure 0 0 0) 0x7f64d4003a00 con 0x7f64dc10d5c0 2024-03-11T19:51:54.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.229+0000 7f64d2a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 0x7f64c00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:54.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.229+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f64d40cd9b0 con 0x7f64dc10d5c0 2024-03-11T19:51:54.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.229+0000 7f64d9e00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 0x7f64c00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:54.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.230+0000 7f64d9e00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 0x7f64c00a4010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009c60 tx=0x7f64cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:54.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.232+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64d4096730 con 0x7f64dc10d5c0 2024-03-11T19:51:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.483+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f64dc19fb10 con 0x7f64dc10d5c0 2024-03-11T19:51:54.489 INFO:teuthology.orchestra.run.smithi107.stdout:343597383717 2024-03-11T19:51:54.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.484+0000 7f64d2a00700 1 -- 172.21.15.107:0/2257500767 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f64d4099df0 con 0x7f64dc10d5c0 2024-03-11T19:51:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.486+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 msgr2=0x7f64c00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.486+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 0x7f64c00a4010 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009c60 tx=0x7f64cc009260 comp rx=0 tx=0).stop 2024-03-11T19:51:54.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.486+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 msgr2=0x7f64dc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.486+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ef30 tx=0x7f64d400c3a0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 shutdown_connections 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f64c00a1b50 0x7f64c00a4010 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64dc10d5c0 0x7f64dc19e630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64dc1084e0 0x7f64dc1a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 --2- 172.21.15.107:0/2257500767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64dc108e90 0x7f64dc1a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:54.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 >> 172.21.15.107:0/2257500767 conn(0x7f64dc075440 msgr2=0x7f64dc0767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:54.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 shutdown_connections 2024-03-11T19:51:54.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:54.487+0000 7f64e0a00700 1 -- 172.21.15.107:0/2257500767 wait complete. 2024-03-11T19:51:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:54 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2257500767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:51:54.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:54 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-11T19:51:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:54 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2257500767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:51:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:54 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-11T19:51:54.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:54 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2257500767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:51:54.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:54 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]: dispatch 2024-03-11T19:51:55.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383711 got 343597383717 for osd.11 2024-03-11T19:51:55.379 DEBUG:teuthology.parallel:result is None 2024-03-11T19:51:55.379 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-11T19:51:55.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-11T19:51:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:55 smithi122 ceph-mon[36397]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:51:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-11T19:51:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-11T19:51:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-11T19:51:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-11T19:51:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-11T19:51:55.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:55 smithi145 ceph-mon[36730]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 8]}]': finished 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-11T19:51:55.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:55 smithi107 ceph-mon[31712]: osdmap e93: 12 total, 12 up, 12 in 2024-03-11T19:51:55.838 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:51:56.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:56 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-11T19:51:56.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:56 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-11T19:51:56.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:56 smithi145 ceph-mon[36730]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:51:56.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:56 smithi145 ceph-mon[36730]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:51:56.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:56 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-11T19:51:56.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:56 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-11T19:51:56.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:56 smithi107 ceph-mon[31712]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:51:56.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:56 smithi107 ceph-mon[31712]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:51:57.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:56 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-11T19:51:57.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:56 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-11T19:51:57.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:56 smithi122 ceph-mon[36397]: osdmap e94: 12 total, 12 up, 12 in 2024-03-11T19:51:57.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:56 smithi122 ceph-mon[36397]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-11T19:51:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:57 smithi145 ceph-mon[36730]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:51:57.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:57 smithi145 ceph-mon[36730]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:51:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:57 smithi107 ceph-mon[31712]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:51:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:57 smithi107 ceph-mon[31712]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:51:58.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:57 smithi122 ceph-mon[36397]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:51:58.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:57 smithi122 ceph-mon[36397]: osdmap e95: 12 total, 12 up, 12 in 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 -- 172.21.15.107:0/1982082523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c06a2c0 msgr2=0x7f095c10eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 --2- 172.21.15.107:0/1982082523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c06a2c0 0x7f095c10eb00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f095400b0d0 tx=0x7f095400b3e0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 -- 172.21.15.107:0/1982082523 shutdown_connections 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 --2- 172.21.15.107:0/1982082523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c06a2c0 0x7f095c10eb00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 --2- 172.21.15.107:0/1982082523 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c107cc0 0x7f095c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 --2- 172.21.15.107:0/1982082523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c069920 0x7f095c069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 -- 172.21.15.107:0/1982082523 >> 172.21.15.107:0/1982082523 conn(0x7f095c077aa0 msgr2=0x7f095c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 -- 172.21.15.107:0/1982082523 shutdown_connections 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.157+0000 7f0961600700 1 -- 172.21.15.107:0/1982082523 wait complete. 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.158+0000 7f0961600700 1 Processor -- start 2024-03-11T19:51:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.158+0000 7f0961600700 1 -- start start 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c06a2c0 0x7f095c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c107cc0 0x7f095c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19d160 con 0x7f095c107cc0 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19d2d0 con 0x7f095c06a2c0 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f0961600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19d490 con 0x7f095c069920 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f095aa00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f095a000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c06a2c0 0x7f095c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f095b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c107cc0 0x7f095c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f095aa00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37634/0 (socket says 172.21.15.107:37634) 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.159+0000 7f095aa00700 1 -- 172.21.15.107:0/2384015656 learned_addr learned my addr 172.21.15.107:0/2384015656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 -- 172.21.15.107:0/2384015656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c06a2c0 msgr2=0x7f095c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c06a2c0 0x7f095c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 -- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c107cc0 msgr2=0x7f095c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c107cc0 0x7f095c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 -- 172.21.15.107:0/2384015656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009d00 con 0x7f095c069920 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f095aa00700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9a0 tx=0x7f094c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c009790 con 0x7f095c069920 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f094c010430 con 0x7f095c069920 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c19d770 con 0x7f095c069920 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c018680 con 0x7f095c069920 2024-03-11T19:51:58.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.160+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c100d40 con 0x7f095c069920 2024-03-11T19:51:58.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.162+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c10a1e0 con 0x7f095c069920 2024-03-11T19:51:58.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.164+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f094c010910 con 0x7f095c069920 2024-03-11T19:51:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.165+0000 7f0953400700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 0x7f09400a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.165+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f094c014070 con 0x7f095c069920 2024-03-11T19:51:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.165+0000 7f095a000700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 0x7f09400a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:58.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.166+0000 7f095a000700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 0x7f09400a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f095c078080 tx=0x7f0948012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:58.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.169+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f094c052d20 con 0x7f095c069920 2024-03-11T19:51:58.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.386+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f095c1995e0 con 0x7f09400a1ac0 2024-03-11T19:51:58.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.403+0000 7f0953400700 1 -- 172.21.15.107:0/2384015656 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340912 (secure 0 0 0) 0x7f095c1995e0 con 0x7f09400a1ac0 2024-03-11T19:51:58.414 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.409+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 msgr2=0x7f09400a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.409+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 0x7f09400a3f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f095c078080 tx=0x7f0948012040 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.409+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 msgr2=0x7f095c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.409+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9a0 tx=0x7f094c00ecb0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 shutdown_connections 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f09400a1ac0 0x7f09400a3f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f095c107cc0 0x7f095c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f095c069920 0x7f095c198280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 --2- 172.21.15.107:0/2384015656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095c06a2c0 0x7f095c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:58.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 >> 172.21.15.107:0/2384015656 conn(0x7f095c077aa0 msgr2=0x7f095c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 shutdown_connections 2024-03-11T19:51:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:51:58.410+0000 7f0961600700 1 -- 172.21.15.107:0/2384015656 wait complete. 2024-03-11T19:51:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:51:58.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:58 smithi107 ceph-mon[31712]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-11T19:51:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:51:58 smithi107 ceph-mon[31712]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:51:59.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:58 smithi122 ceph-mon[36397]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-11T19:51:59.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:51:58 smithi122 ceph-mon[36397]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:51:59.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:58 smithi145 ceph-mon[36730]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-11T19:51:59.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:51:58 smithi145 ceph-mon[36730]: from='client.25060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:51:59.367 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-03-11T19:51:58.376137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657159,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":679,"num_read_kb":465,"num_write":354,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182000,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":744,"ondisk_log_size":744,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353372,"kb_used_data":12476,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588452,"statfs":{"total":1151940427776,"available":1151578574848,"internally_reserved":0,"allocated":12775424,"data_stored":8206545,"data_compressed":407979,"data_compressed_allocated":557056,"data_compressed_original":2613248,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":27,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002058"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709765+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:51:56.709765+0000","last_peered":"2024-03-11T19:51:56.709765+0000","last_clean":"2024-03-11T19:51:56.709765+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:51:56.709765+0000","last_undegraded":"2024-03-11T19:51:56.709765+0000","last_fullsized":"2024-03-11T19:51:56.709765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.525207+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:51:55.525207+0000","last_peered":"2024-03-11T19:51:55.525207+0000","last_clean":"2024-03-11T19:51:55.525207+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:51:55.525207+0000","last_undegraded":"2024-03-11T19:51:55.525207+0000","last_fullsized":"2024-03-11T19:51:55.525207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.524067+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:51:55.524067+0000","last_peered":"2024-03-11T19:51:55.524067+0000","last_clean":"2024-03-11T19:51:55.524067+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:51:55.524067+0000","last_undegraded":"2024-03-11T19:51:55.524067+0000","last_fullsized":"2024-03-11T19:51:55.524067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501964+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:51:54.501964+0000","last_peered":"2024-03-11T19:51:54.501964+0000","last_clean":"2024-03-11T19:51:54.501964+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:51:54.501964+0000","last_undegraded":"2024-03-11T19:51:54.501964+0000","last_fullsized":"2024-03-11T19:51:54.501964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507584+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:51:54.507584+0000","last_peered":"2024-03-11T19:51:54.507584+0000","last_clean":"2024-03-11T19:51:54.507584+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:51:54.507584+0000","last_undegraded":"2024-03-11T19:51:54.507584+0000","last_fullsized":"2024-03-11T19:51:54.507584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532313+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:51:56.532313+0000","last_peered":"2024-03-11T19:51:56.532313+0000","last_clean":"2024-03-11T19:51:56.532313+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:51:56.532313+0000","last_undegraded":"2024-03-11T19:51:56.532313+0000","last_fullsized":"2024-03-11T19:51:56.532313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507506+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:51:54.507506+0000","last_peered":"2024-03-11T19:51:54.507506+0000","last_clean":"2024-03-11T19:51:54.507506+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:51:54.507506+0000","last_undegraded":"2024-03-11T19:51:54.507506+0000","last_fullsized":"2024-03-11T19:51:54.507506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.524741+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:51:56.524741+0000","last_peered":"2024-03-11T19:51:56.524741+0000","last_clean":"2024-03-11T19:51:56.524741+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:51:56.524741+0000","last_undegraded":"2024-03-11T19:51:56.524741+0000","last_fullsized":"2024-03-11T19:51:56.524741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709398+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:51:56.709398+0000","last_peered":"2024-03-11T19:51:56.709398+0000","last_clean":"2024-03-11T19:51:56.709398+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:51:56.709398+0000","last_undegraded":"2024-03-11T19:51:56.709398+0000","last_fullsized":"2024-03-11T19:51:56.709398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532578+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:51:56.532578+0000","last_peered":"2024-03-11T19:51:56.532578+0000","last_clean":"2024-03-11T19:51:56.532578+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:51:56.532578+0000","last_undegraded":"2024-03-11T19:51:56.532578+0000","last_fullsized":"2024-03-11T19:51:56.532578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507999+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:51:54.507999+0000","last_peered":"2024-03-11T19:51:54.507999+0000","last_clean":"2024-03-11T19:51:54.507999+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:51:54.507999+0000","last_undegraded":"2024-03-11T19:51:54.507999+0000","last_fullsized":"2024-03-11T19:51:54.507999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501831+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:51:54.501831+0000","last_peered":"2024-03-11T19:51:54.501831+0000","last_clean":"2024-03-11T19:51:54.501831+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:51:54.501831+0000","last_undegraded":"2024-03-11T19:51:54.501831+0000","last_fullsized":"2024-03-11T19:51:54.501831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.317487+0000","last_change":"2024-03-11T19:49:55.201525+0000","last_active":"2024-03-11T19:51:54.317487+0000","last_peered":"2024-03-11T19:51:54.317487+0000","last_clean":"2024-03-11T19:51:54.317487+0000","last_became_active":"2024-03-11T19:49:55.199729+0000","last_became_peered":"2024-03-11T19:49:55.199729+0000","last_unstale":"2024-03-11T19:51:54.317487+0000","last_undegraded":"2024-03-11T19:51:54.317487+0000","last_fullsized":"2024-03-11T19:51:54.317487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:03:26.753822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523062+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:51:55.523062+0000","last_peered":"2024-03-11T19:51:55.523062+0000","last_clean":"2024-03-11T19:51:55.523062+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:51:55.523062+0000","last_undegraded":"2024-03-11T19:51:55.523062+0000","last_fullsized":"2024-03-11T19:51:55.523062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.522612+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:51:55.522612+0000","last_peered":"2024-03-11T19:51:55.522612+0000","last_clean":"2024-03-11T19:51:55.522612+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:51:55.522612+0000","last_undegraded":"2024-03-11T19:51:55.522612+0000","last_fullsized":"2024-03-11T19:51:55.522612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709009+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:51:56.709009+0000","last_peered":"2024-03-11T19:51:56.709009+0000","last_clean":"2024-03-11T19:51:56.709009+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:51:56.709009+0000","last_undegraded":"2024-03-11T19:51:56.709009+0000","last_fullsized":"2024-03-11T19:51:56.709009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709723+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:51:56.709723+0000","last_peered":"2024-03-11T19:51:56.709723+0000","last_clean":"2024-03-11T19:51:56.709723+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:51:56.709723+0000","last_undegraded":"2024-03-11T19:51:56.709723+0000","last_fullsized":"2024-03-11T19:51:56.709723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514991+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:51:54.514991+0000","last_peered":"2024-03-11T19:51:54.514991+0000","last_clean":"2024-03-11T19:51:54.514991+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:51:54.514991+0000","last_undegraded":"2024-03-11T19:51:54.514991+0000","last_fullsized":"2024-03-11T19:51:54.514991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617426+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:51:54.617426+0000","last_peered":"2024-03-11T19:51:54.617426+0000","last_clean":"2024-03-11T19:51:54.617426+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:51:54.617426+0000","last_undegraded":"2024-03-11T19:51:54.617426+0000","last_fullsized":"2024-03-11T19:51:54.617426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617239+0000","last_change":"2024-03-11T19:50:55.487486+0000","last_active":"2024-03-11T19:51:54.617239+0000","last_peered":"2024-03-11T19:51:54.617239+0000","last_clean":"2024-03-11T19:51:54.617239+0000","last_became_active":"2024-03-11T19:50:55.486998+0000","last_became_peered":"2024-03-11T19:50:55.486998+0000","last_unstale":"2024-03-11T19:51:54.617239+0000","last_undegraded":"2024-03-11T19:51:54.617239+0000","last_fullsized":"2024-03-11T19:51:54.617239+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:39.904812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.515063+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:51:54.515063+0000","last_peered":"2024-03-11T19:51:54.515063+0000","last_clean":"2024-03-11T19:51:54.515063+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:51:54.515063+0000","last_undegraded":"2024-03-11T19:51:54.515063+0000","last_fullsized":"2024-03-11T19:51:54.515063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526502+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:51:56.526502+0000","last_peered":"2024-03-11T19:51:56.526502+0000","last_clean":"2024-03-11T19:51:56.526502+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:51:56.526502+0000","last_undegraded":"2024-03-11T19:51:56.526502+0000","last_fullsized":"2024-03-11T19:51:56.526502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501383+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:51:54.501383+0000","last_peered":"2024-03-11T19:51:54.501383+0000","last_clean":"2024-03-11T19:51:54.501383+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:51:54.501383+0000","last_undegraded":"2024-03-11T19:51:54.501383+0000","last_fullsized":"2024-03-11T19:51:54.501383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527995+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:51:56.527995+0000","last_peered":"2024-03-11T19:51:56.527995+0000","last_clean":"2024-03-11T19:51:56.527995+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:51:56.527995+0000","last_undegraded":"2024-03-11T19:51:56.527995+0000","last_fullsized":"2024-03-11T19:51:56.527995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.528111+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:51:55.528111+0000","last_peered":"2024-03-11T19:51:55.528111+0000","last_clean":"2024-03-11T19:51:55.528111+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:51:55.528111+0000","last_undegraded":"2024-03-11T19:51:55.528111+0000","last_fullsized":"2024-03-11T19:51:55.528111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501559+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:51:54.501559+0000","last_peered":"2024-03-11T19:51:54.501559+0000","last_clean":"2024-03-11T19:51:54.501559+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:51:54.501559+0000","last_undegraded":"2024-03-11T19:51:54.501559+0000","last_fullsized":"2024-03-11T19:51:54.501559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709813+0000","last_change":"2024-03-11T19:50:55.480971+0000","last_active":"2024-03-11T19:51:56.709813+0000","last_peered":"2024-03-11T19:51:56.709813+0000","last_clean":"2024-03-11T19:51:56.709813+0000","last_became_active":"2024-03-11T19:50:55.474612+0000","last_became_peered":"2024-03-11T19:50:55.474612+0000","last_unstale":"2024-03-11T19:51:56.709813+0000","last_undegraded":"2024-03-11T19:51:56.709813+0000","last_fullsized":"2024-03-11T19:51:56.709813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:59:47.829059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501410+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:51:54.501410+0000","last_peered":"2024-03-11T19:51:54.501410+0000","last_clean":"2024-03-11T19:51:54.501410+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:51:54.501410+0000","last_undegraded":"2024-03-11T19:51:54.501410+0000","last_fullsized":"2024-03-11T19:51:54.501410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507717+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:51:54.507717+0000","last_peered":"2024-03-11T19:51:54.507717+0000","last_clean":"2024-03-11T19:51:54.507717+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:51:54.507717+0000","last_undegraded":"2024-03-11T19:51:54.507717+0000","last_fullsized":"2024-03-11T19:51:54.507717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.473698+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:50:55.473698+0000","last_peered":"2024-03-11T19:50:55.473698+0000","last_clean":"2024-03-11T19:50:55.473698+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:50:55.473698+0000","last_undegraded":"2024-03-11T19:50:55.473698+0000","last_fullsized":"2024-03-11T19:50:55.473698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617360+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:51:54.617360+0000","last_peered":"2024-03-11T19:51:54.617360+0000","last_clean":"2024-03-11T19:51:54.617360+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:51:54.617360+0000","last_undegraded":"2024-03-11T19:51:54.617360+0000","last_fullsized":"2024-03-11T19:51:54.617360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:51.584419+0000","last_change":"2024-03-11T19:50:55.509345+0000","last_active":"2024-03-11T19:51:51.584419+0000","last_peered":"2024-03-11T19:51:51.584419+0000","last_clean":"2024-03-11T19:51:51.584419+0000","last_became_active":"2024-03-11T19:50:55.505792+0000","last_became_peered":"2024-03-11T19:50:55.505792+0000","last_unstale":"2024-03-11T19:51:51.584419+0000","last_undegraded":"2024-03-11T19:51:51.584419+0000","last_fullsized":"2024-03-11T19:51:51.584419+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:01:20.682572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507546+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:51:54.507546+0000","last_peered":"2024-03-11T19:51:54.507546+0000","last_clean":"2024-03-11T19:51:54.507546+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:51:54.507546+0000","last_undegraded":"2024-03-11T19:51:54.507546+0000","last_fullsized":"2024-03-11T19:51:54.507546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617164+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:51:54.617164+0000","last_peered":"2024-03-11T19:51:54.617164+0000","last_clean":"2024-03-11T19:51:54.617164+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:51:54.617164+0000","last_undegraded":"2024-03-11T19:51:54.617164+0000","last_fullsized":"2024-03-11T19:51:54.617164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532197+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:51:56.532197+0000","last_peered":"2024-03-11T19:51:56.532197+0000","last_clean":"2024-03-11T19:51:56.532197+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:51:56.532197+0000","last_undegraded":"2024-03-11T19:51:56.532197+0000","last_fullsized":"2024-03-11T19:51:56.532197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.474133+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:50:55.474133+0000","last_peered":"2024-03-11T19:50:55.474133+0000","last_clean":"2024-03-11T19:50:55.474133+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:50:55.474133+0000","last_undegraded":"2024-03-11T19:50:55.474133+0000","last_fullsized":"2024-03-11T19:50:55.474133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.468325+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:50:55.468325+0000","last_peered":"2024-03-11T19:50:55.468325+0000","last_clean":"2024-03-11T19:50:55.468325+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:50:55.468325+0000","last_undegraded":"2024-03-11T19:50:55.468325+0000","last_fullsized":"2024-03-11T19:50:55.468325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":2,"reported_epoch":93,"state":"peering","last_fresh":"2024-03-11T19:51:54.325327+0000","last_change":"2024-03-11T19:51:55.398160+0000","last_active":"2024-03-11T19:51:55.398160+0000","last_peered":"2024-03-11T19:51:54.325327+0000","last_clean":"2024-03-11T19:51:54.325327+0000","last_became_active":"2024-03-11T19:49:55.236259+0000","last_became_peered":"2024-03-11T19:49:55.236259+0000","last_unstale":"2024-03-11T19:51:54.325327+0000","last_undegraded":"2024-03-11T19:51:54.325327+0000","last_fullsized":"2024-03-11T19:51:54.325327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.524925+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:51:55.524925+0000","last_peered":"2024-03-11T19:51:55.524925+0000","last_clean":"2024-03-11T19:51:55.524925+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:51:55.524925+0000","last_undegraded":"2024-03-11T19:51:55.524925+0000","last_fullsized":"2024-03-11T19:51:55.524925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472476+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:50:55.472476+0000","last_peered":"2024-03-11T19:50:55.472476+0000","last_clean":"2024-03-11T19:50:55.472476+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:50:55.472476+0000","last_undegraded":"2024-03-11T19:50:55.472476+0000","last_fullsized":"2024-03-11T19:50:55.472476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513839+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:51:54.513839+0000","last_peered":"2024-03-11T19:51:54.513839+0000","last_clean":"2024-03-11T19:51:54.513839+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:51:54.513839+0000","last_undegraded":"2024-03-11T19:51:54.513839+0000","last_fullsized":"2024-03-11T19:51:54.513839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525570+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:51:56.525570+0000","last_peered":"2024-03-11T19:51:56.525570+0000","last_clean":"2024-03-11T19:51:56.525570+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:51:56.525570+0000","last_undegraded":"2024-03-11T19:51:56.525570+0000","last_fullsized":"2024-03-11T19:51:56.525570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.594032+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:51:56.594032+0000","last_peered":"2024-03-11T19:51:56.594032+0000","last_clean":"2024-03-11T19:51:56.594032+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:51:56.594032+0000","last_undegraded":"2024-03-11T19:51:56.594032+0000","last_fullsized":"2024-03-11T19:51:56.594032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709914+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:51:56.709914+0000","last_peered":"2024-03-11T19:51:56.709914+0000","last_clean":"2024-03-11T19:51:56.709914+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:51:56.709914+0000","last_undegraded":"2024-03-11T19:51:56.709914+0000","last_fullsized":"2024-03-11T19:51:56.709914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513462+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:51:54.513462+0000","last_peered":"2024-03-11T19:51:54.513462+0000","last_clean":"2024-03-11T19:51:54.513462+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:51:54.513462+0000","last_undegraded":"2024-03-11T19:51:54.513462+0000","last_fullsized":"2024-03-11T19:51:54.513462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507842+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:51:54.507842+0000","last_peered":"2024-03-11T19:51:54.507842+0000","last_clean":"2024-03-11T19:51:54.507842+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:51:54.507842+0000","last_undegraded":"2024-03-11T19:51:54.507842+0000","last_fullsized":"2024-03-11T19:51:54.507842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.547076+0000","last_change":"2024-03-11T19:51:55.547076+0000","last_active":"2024-03-11T19:51:55.547076+0000","last_peered":"2024-03-11T19:51:55.547076+0000","last_clean":"2024-03-11T19:51:55.547076+0000","last_became_active":"2024-03-11T19:51:55.546410+0000","last_became_peered":"2024-03-11T19:51:55.546410+0000","last_unstale":"2024-03-11T19:51:55.547076+0000","last_undegraded":"2024-03-11T19:51:55.547076+0000","last_fullsized":"2024-03-11T19:51:55.547076+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:43:21.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501716+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:51:54.501716+0000","last_peered":"2024-03-11T19:51:54.501716+0000","last_clean":"2024-03-11T19:51:54.501716+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:51:54.501716+0000","last_undegraded":"2024-03-11T19:51:54.501716+0000","last_fullsized":"2024-03-11T19:51:54.501716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.473576+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:50:55.473576+0000","last_peered":"2024-03-11T19:50:55.473576+0000","last_clean":"2024-03-11T19:50:55.473576+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:50:55.473576+0000","last_undegraded":"2024-03-11T19:50:55.473576+0000","last_fullsized":"2024-03-11T19:50:55.473576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532035+0000","last_change":"2024-03-11T19:50:55.475760+0000","last_active":"2024-03-11T19:51:56.532035+0000","last_peered":"2024-03-11T19:51:56.532035+0000","last_clean":"2024-03-11T19:51:56.532035+0000","last_became_active":"2024-03-11T19:50:55.475170+0000","last_became_peered":"2024-03-11T19:50:55.475170+0000","last_unstale":"2024-03-11T19:51:56.532035+0000","last_undegraded":"2024-03-11T19:51:56.532035+0000","last_fullsized":"2024-03-11T19:51:56.532035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:58.278212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":252,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:57.597589+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:51:57.597589+0000","last_peered":"2024-03-11T19:51:57.597589+0000","last_clean":"2024-03-11T19:51:57.597589+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:51:57.597589+0000","last_undegraded":"2024-03-11T19:51:57.597589+0000","last_fullsized":"2024-03-11T19:51:57.597589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":153,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507749+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:51:54.507749+0000","last_peered":"2024-03-11T19:51:54.507749+0000","last_clean":"2024-03-11T19:51:54.507749+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:51:54.507749+0000","last_undegraded":"2024-03-11T19:51:54.507749+0000","last_fullsized":"2024-03-11T19:51:54.507749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.473984+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:50:55.473984+0000","last_peered":"2024-03-11T19:50:55.473984+0000","last_clean":"2024-03-11T19:50:55.473984+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:50:55.473984+0000","last_undegraded":"2024-03-11T19:50:55.473984+0000","last_fullsized":"2024-03-11T19:50:55.473984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617271+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:51:54.617271+0000","last_peered":"2024-03-11T19:51:54.617271+0000","last_clean":"2024-03-11T19:51:54.617271+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:51:54.617271+0000","last_undegraded":"2024-03-11T19:51:54.617271+0000","last_fullsized":"2024-03-11T19:51:54.617271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.311699+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:51:54.311699+0000","last_peered":"2024-03-11T19:51:54.311699+0000","last_clean":"2024-03-11T19:51:54.311699+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:51:54.311699+0000","last_undegraded":"2024-03-11T19:51:54.311699+0000","last_fullsized":"2024-03-11T19:51:54.311699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709442+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:51:56.709442+0000","last_peered":"2024-03-11T19:51:56.709442+0000","last_clean":"2024-03-11T19:51:56.709442+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:51:56.709442+0000","last_undegraded":"2024-03-11T19:51:56.709442+0000","last_fullsized":"2024-03-11T19:51:56.709442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.473677+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:50:55.473677+0000","last_peered":"2024-03-11T19:50:55.473677+0000","last_clean":"2024-03-11T19:50:55.473677+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:50:55.473677+0000","last_undegraded":"2024-03-11T19:50:55.473677+0000","last_fullsized":"2024-03-11T19:50:55.473677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507895+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:51:54.507895+0000","last_peered":"2024-03-11T19:51:54.507895+0000","last_clean":"2024-03-11T19:51:54.507895+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:51:54.507895+0000","last_undegraded":"2024-03-11T19:51:54.507895+0000","last_fullsized":"2024-03-11T19:51:54.507895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514760+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:51:54.514760+0000","last_peered":"2024-03-11T19:51:54.514760+0000","last_clean":"2024-03-11T19:51:54.514760+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:51:54.514760+0000","last_undegraded":"2024-03-11T19:51:54.514760+0000","last_fullsized":"2024-03-11T19:51:54.514760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.629348+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:51:54.629348+0000","last_peered":"2024-03-11T19:51:54.629348+0000","last_clean":"2024-03-11T19:51:54.629348+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:51:54.629348+0000","last_undegraded":"2024-03-11T19:51:54.629348+0000","last_fullsized":"2024-03-11T19:51:54.629348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507512+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:51:54.507512+0000","last_peered":"2024-03-11T19:51:54.507512+0000","last_clean":"2024-03-11T19:51:54.507512+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:51:54.507512+0000","last_undegraded":"2024-03-11T19:51:54.507512+0000","last_fullsized":"2024-03-11T19:51:54.507512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.470162+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:50:55.470162+0000","last_peered":"2024-03-11T19:50:55.470162+0000","last_clean":"2024-03-11T19:50:55.470162+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:50:55.470162+0000","last_undegraded":"2024-03-11T19:50:55.470162+0000","last_fullsized":"2024-03-11T19:50:55.470162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":71,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507389+0000","last_change":"2024-03-11T19:50:55.807772+0000","last_active":"2024-03-11T19:51:54.507389+0000","last_peered":"2024-03-11T19:51:54.507389+0000","last_clean":"2024-03-11T19:51:54.507389+0000","last_became_active":"2024-03-11T19:50:55.794019+0000","last_became_peered":"2024-03-11T19:50:55.794019+0000","last_unstale":"2024-03-11T19:51:54.507389+0000","last_undegraded":"2024-03-11T19:51:54.507389+0000","last_fullsized":"2024-03-11T19:51:54.507389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:00.170150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.793868+0000","last_change":"2024-03-11T19:50:55.793868+0000","last_active":"2024-03-11T19:50:55.793868+0000","last_peered":"2024-03-11T19:50:55.793868+0000","last_clean":"2024-03-11T19:50:55.793868+0000","last_became_active":"2024-03-11T19:50:55.793299+0000","last_became_peered":"2024-03-11T19:50:55.793299+0000","last_unstale":"2024-03-11T19:50:55.793868+0000","last_undegraded":"2024-03-11T19:50:55.793868+0000","last_fullsized":"2024-03-11T19:50:55.793868+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:40:42.039449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472404+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:50:55.472404+0000","last_peered":"2024-03-11T19:50:55.472404+0000","last_clean":"2024-03-11T19:50:55.472404+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:50:55.472404+0000","last_undegraded":"2024-03-11T19:50:55.472404+0000","last_fullsized":"2024-03-11T19:50:55.472404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507913+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:51:54.507913+0000","last_peered":"2024-03-11T19:51:54.507913+0000","last_clean":"2024-03-11T19:51:54.507913+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:51:54.507913+0000","last_undegraded":"2024-03-11T19:51:54.507913+0000","last_fullsized":"2024-03-11T19:51:54.507913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709559+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:51:56.709559+0000","last_peered":"2024-03-11T19:51:56.709559+0000","last_clean":"2024-03-11T19:51:56.709559+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:51:56.709559+0000","last_undegraded":"2024-03-11T19:51:56.709559+0000","last_fullsized":"2024-03-11T19:51:56.709559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507572+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:51:54.507572+0000","last_peered":"2024-03-11T19:51:54.507572+0000","last_clean":"2024-03-11T19:51:54.507572+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:51:54.507572+0000","last_undegraded":"2024-03-11T19:51:54.507572+0000","last_fullsized":"2024-03-11T19:51:54.507572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513579+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:51:54.513579+0000","last_peered":"2024-03-11T19:51:54.513579+0000","last_clean":"2024-03-11T19:51:54.513579+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:51:54.513579+0000","last_undegraded":"2024-03-11T19:51:54.513579+0000","last_fullsized":"2024-03-11T19:51:54.513579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532533+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:51:56.532533+0000","last_peered":"2024-03-11T19:51:56.532533+0000","last_clean":"2024-03-11T19:51:56.532533+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:51:56.532533+0000","last_undegraded":"2024-03-11T19:51:56.532533+0000","last_fullsized":"2024-03-11T19:51:56.532533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.468834+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:50:55.468834+0000","last_peered":"2024-03-11T19:50:55.468834+0000","last_clean":"2024-03-11T19:50:55.468834+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:50:55.468834+0000","last_undegraded":"2024-03-11T19:50:55.468834+0000","last_fullsized":"2024-03-11T19:50:55.468834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507584+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:51:54.507584+0000","last_peered":"2024-03-11T19:51:54.507584+0000","last_clean":"2024-03-11T19:51:54.507584+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:51:54.507584+0000","last_undegraded":"2024-03-11T19:51:54.507584+0000","last_fullsized":"2024-03-11T19:51:54.507584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532656+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:51:56.532656+0000","last_peered":"2024-03-11T19:51:56.532656+0000","last_clean":"2024-03-11T19:51:56.532656+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:51:56.532656+0000","last_undegraded":"2024-03-11T19:51:56.532656+0000","last_fullsized":"2024-03-11T19:51:56.532656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.470525+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:50:55.470525+0000","last_peered":"2024-03-11T19:50:55.470525+0000","last_clean":"2024-03-11T19:50:55.470525+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:50:55.470525+0000","last_undegraded":"2024-03-11T19:50:55.470525+0000","last_fullsized":"2024-03-11T19:50:55.470525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472712+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:50:55.472712+0000","last_peered":"2024-03-11T19:50:55.472712+0000","last_clean":"2024-03-11T19:50:55.472712+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:50:55.472712+0000","last_undegraded":"2024-03-11T19:50:55.472712+0000","last_fullsized":"2024-03-11T19:50:55.472712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:53.626043+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:51:53.626043+0000","last_peered":"2024-03-11T19:51:53.626043+0000","last_clean":"2024-03-11T19:51:53.626043+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:51:53.626043+0000","last_undegraded":"2024-03-11T19:51:53.626043+0000","last_fullsized":"2024-03-11T19:51:53.626043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507895+0000","last_change":"2024-03-11T19:50:55.488155+0000","last_active":"2024-03-11T19:51:54.507895+0000","last_peered":"2024-03-11T19:51:54.507895+0000","last_clean":"2024-03-11T19:51:54.507895+0000","last_became_active":"2024-03-11T19:50:55.477254+0000","last_became_peered":"2024-03-11T19:50:55.477254+0000","last_unstale":"2024-03-11T19:51:54.507895+0000","last_undegraded":"2024-03-11T19:51:54.507895+0000","last_fullsized":"2024-03-11T19:51:54.507895+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:05.827200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527168+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:51:56.527168+0000","last_peered":"2024-03-11T19:51:56.527168+0000","last_clean":"2024-03-11T19:51:56.527168+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:51:56.527168+0000","last_undegraded":"2024-03-11T19:51:56.527168+0000","last_fullsized":"2024-03-11T19:51:56.527168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501721+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:51:54.501721+0000","last_peered":"2024-03-11T19:51:54.501721+0000","last_clean":"2024-03-11T19:51:54.501721+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:51:54.501721+0000","last_undegraded":"2024-03-11T19:51:54.501721+0000","last_fullsized":"2024-03-11T19:51:54.501721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507397+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:51:54.507397+0000","last_peered":"2024-03-11T19:51:54.507397+0000","last_clean":"2024-03-11T19:51:54.507397+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:51:54.507397+0000","last_undegraded":"2024-03-11T19:51:54.507397+0000","last_fullsized":"2024-03-11T19:51:54.507397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472270+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:50:55.472270+0000","last_peered":"2024-03-11T19:50:55.472270+0000","last_clean":"2024-03-11T19:50:55.472270+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:50:55.472270+0000","last_undegraded":"2024-03-11T19:50:55.472270+0000","last_fullsized":"2024-03-11T19:50:55.472270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.307361+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:51:54.307361+0000","last_peered":"2024-03-11T19:51:54.307361+0000","last_clean":"2024-03-11T19:51:54.307361+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:51:54.307361+0000","last_undegraded":"2024-03-11T19:51:54.307361+0000","last_fullsized":"2024-03-11T19:51:54.307361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507542+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:51:54.507542+0000","last_peered":"2024-03-11T19:51:54.507542+0000","last_clean":"2024-03-11T19:51:54.507542+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:51:54.507542+0000","last_undegraded":"2024-03-11T19:51:54.507542+0000","last_fullsized":"2024-03-11T19:51:54.507542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709301+0000","last_change":"2024-03-11T19:50:55.474853+0000","last_active":"2024-03-11T19:51:56.709301+0000","last_peered":"2024-03-11T19:51:56.709301+0000","last_clean":"2024-03-11T19:51:56.709301+0000","last_became_active":"2024-03-11T19:50:55.474379+0000","last_became_peered":"2024-03-11T19:50:55.474379+0000","last_unstale":"2024-03-11T19:51:56.709301+0000","last_undegraded":"2024-03-11T19:51:56.709301+0000","last_fullsized":"2024-03-11T19:51:56.709301+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:43.691094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709662+0000","last_change":"2024-03-11T19:50:55.491848+0000","last_active":"2024-03-11T19:51:56.709662+0000","last_peered":"2024-03-11T19:51:56.709662+0000","last_clean":"2024-03-11T19:51:56.709662+0000","last_became_active":"2024-03-11T19:50:55.485786+0000","last_became_peered":"2024-03-11T19:50:55.485786+0000","last_unstale":"2024-03-11T19:51:56.709662+0000","last_undegraded":"2024-03-11T19:51:56.709662+0000","last_fullsized":"2024-03-11T19:51:56.709662+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:24:41.406176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.315130+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:51:54.315130+0000","last_peered":"2024-03-11T19:51:54.315130+0000","last_clean":"2024-03-11T19:51:54.315130+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:51:54.315130+0000","last_undegraded":"2024-03-11T19:51:54.315130+0000","last_fullsized":"2024-03-11T19:51:54.315130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525078+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:51:56.525078+0000","last_peered":"2024-03-11T19:51:56.525078+0000","last_clean":"2024-03-11T19:51:56.525078+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:51:56.525078+0000","last_undegraded":"2024-03-11T19:51:56.525078+0000","last_fullsized":"2024-03-11T19:51:56.525078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514897+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:51:54.514897+0000","last_peered":"2024-03-11T19:51:54.514897+0000","last_clean":"2024-03-11T19:51:54.514897+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:51:54.514897+0000","last_undegraded":"2024-03-11T19:51:54.514897+0000","last_fullsized":"2024-03-11T19:51:54.514897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526601+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:51:56.526601+0000","last_peered":"2024-03-11T19:51:56.526601+0000","last_clean":"2024-03-11T19:51:56.526601+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:51:56.526601+0000","last_undegraded":"2024-03-11T19:51:56.526601+0000","last_fullsized":"2024-03-11T19:51:56.526601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.616871+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:51:54.616871+0000","last_peered":"2024-03-11T19:51:54.616871+0000","last_clean":"2024-03-11T19:51:54.616871+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:51:54.616871+0000","last_undegraded":"2024-03-11T19:51:54.616871+0000","last_fullsized":"2024-03-11T19:51:54.616871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501966+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:51:54.501966+0000","last_peered":"2024-03-11T19:51:54.501966+0000","last_clean":"2024-03-11T19:51:54.501966+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:51:54.501966+0000","last_undegraded":"2024-03-11T19:51:54.501966+0000","last_fullsized":"2024-03-11T19:51:54.501966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.525319+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:51:55.525319+0000","last_peered":"2024-03-11T19:51:55.525319+0000","last_clean":"2024-03-11T19:51:55.525319+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:51:55.525319+0000","last_undegraded":"2024-03-11T19:51:55.525319+0000","last_fullsized":"2024-03-11T19:51:55.525319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501740+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:51:54.501740+0000","last_peered":"2024-03-11T19:51:54.501740+0000","last_clean":"2024-03-11T19:51:54.501740+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:51:54.501740+0000","last_undegraded":"2024-03-11T19:51:54.501740+0000","last_fullsized":"2024-03-11T19:51:54.501740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709061+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:51:56.709061+0000","last_peered":"2024-03-11T19:51:56.709061+0000","last_clean":"2024-03-11T19:51:56.709061+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:51:56.709061+0000","last_undegraded":"2024-03-11T19:51:56.709061+0000","last_fullsized":"2024-03-11T19:51:56.709061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507460+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:51:54.507460+0000","last_peered":"2024-03-11T19:51:54.507460+0000","last_clean":"2024-03-11T19:51:54.507460+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:51:54.507460+0000","last_undegraded":"2024-03-11T19:51:54.507460+0000","last_fullsized":"2024-03-11T19:51:54.507460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514956+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:51:54.514956+0000","last_peered":"2024-03-11T19:51:54.514956+0000","last_clean":"2024-03-11T19:51:54.514956+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:51:54.514956+0000","last_undegraded":"2024-03-11T19:51:54.514956+0000","last_fullsized":"2024-03-11T19:51:54.514956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525797+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:51:56.525797+0000","last_peered":"2024-03-11T19:51:56.525797+0000","last_clean":"2024-03-11T19:51:56.525797+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:51:56.525797+0000","last_undegraded":"2024-03-11T19:51:56.525797+0000","last_fullsized":"2024-03-11T19:51:56.525797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709514+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:51:56.709514+0000","last_peered":"2024-03-11T19:51:56.709514+0000","last_clean":"2024-03-11T19:51:56.709514+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:51:56.709514+0000","last_undegraded":"2024-03-11T19:51:56.709514+0000","last_fullsized":"2024-03-11T19:51:56.709514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501705+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:51:54.501705+0000","last_peered":"2024-03-11T19:51:54.501705+0000","last_clean":"2024-03-11T19:51:54.501705+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:51:54.501705+0000","last_undegraded":"2024-03-11T19:51:54.501705+0000","last_fullsized":"2024-03-11T19:51:54.501705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617068+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:51:54.617068+0000","last_peered":"2024-03-11T19:51:54.617068+0000","last_clean":"2024-03-11T19:51:54.617068+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:51:54.617068+0000","last_undegraded":"2024-03-11T19:51:54.617068+0000","last_fullsized":"2024-03-11T19:51:54.617068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525612+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:51:56.525612+0000","last_peered":"2024-03-11T19:51:56.525612+0000","last_clean":"2024-03-11T19:51:56.525612+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:51:56.525612+0000","last_undegraded":"2024-03-11T19:51:56.525612+0000","last_fullsized":"2024-03-11T19:51:56.525612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.501734+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:51:54.501734+0000","last_peered":"2024-03-11T19:51:54.501734+0000","last_clean":"2024-03-11T19:51:54.501734+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:51:54.501734+0000","last_undegraded":"2024-03-11T19:51:54.501734+0000","last_fullsized":"2024-03-11T19:51:54.501734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507554+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:51:54.507554+0000","last_peered":"2024-03-11T19:51:54.507554+0000","last_clean":"2024-03-11T19:51:54.507554+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:51:54.507554+0000","last_undegraded":"2024-03-11T19:51:54.507554+0000","last_fullsized":"2024-03-11T19:51:54.507554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507757+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:51:54.507757+0000","last_peered":"2024-03-11T19:51:54.507757+0000","last_clean":"2024-03-11T19:51:54.507757+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:51:54.507757+0000","last_undegraded":"2024-03-11T19:51:54.507757+0000","last_fullsized":"2024-03-11T19:51:54.507757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708833+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:51:56.708833+0000","last_peered":"2024-03-11T19:51:56.708833+0000","last_clean":"2024-03-11T19:51:56.708833+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:51:56.708833+0000","last_undegraded":"2024-03-11T19:51:56.708833+0000","last_fullsized":"2024-03-11T19:51:56.708833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617504+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:51:54.617504+0000","last_peered":"2024-03-11T19:51:54.617504+0000","last_clean":"2024-03-11T19:51:54.617504+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:51:54.617504+0000","last_undegraded":"2024-03-11T19:51:54.617504+0000","last_fullsized":"2024-03-11T19:51:54.617504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.508052+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:51:54.508052+0000","last_peered":"2024-03-11T19:51:54.508052+0000","last_clean":"2024-03-11T19:51:54.508052+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:51:54.508052+0000","last_undegraded":"2024-03-11T19:51:54.508052+0000","last_fullsized":"2024-03-11T19:51:54.508052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523185+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:51:55.523185+0000","last_peered":"2024-03-11T19:51:55.523185+0000","last_clean":"2024-03-11T19:51:55.523185+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:51:55.523185+0000","last_undegraded":"2024-03-11T19:51:55.523185+0000","last_fullsized":"2024-03-11T19:51:55.523185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.473813+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:50:55.473813+0000","last_peered":"2024-03-11T19:50:55.473813+0000","last_clean":"2024-03-11T19:50:55.473813+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:50:55.473813+0000","last_undegraded":"2024-03-11T19:50:55.473813+0000","last_fullsized":"2024-03-11T19:50:55.473813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507733+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:51:54.507733+0000","last_peered":"2024-03-11T19:51:54.507733+0000","last_clean":"2024-03-11T19:51:54.507733+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:51:54.507733+0000","last_undegraded":"2024-03-11T19:51:54.507733+0000","last_fullsized":"2024-03-11T19:51:54.507733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472261+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:50:55.472261+0000","last_peered":"2024-03-11T19:50:55.472261+0000","last_clean":"2024-03-11T19:50:55.472261+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:50:55.472261+0000","last_undegraded":"2024-03-11T19:50:55.472261+0000","last_fullsized":"2024-03-11T19:50:55.472261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.470132+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:50:55.470132+0000","last_peered":"2024-03-11T19:50:55.470132+0000","last_clean":"2024-03-11T19:50:55.470132+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:50:55.470132+0000","last_undegraded":"2024-03-11T19:50:55.470132+0000","last_fullsized":"2024-03-11T19:50:55.470132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.507908+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:51:54.507908+0000","last_peered":"2024-03-11T19:51:54.507908+0000","last_clean":"2024-03-11T19:51:54.507908+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:51:54.507908+0000","last_undegraded":"2024-03-11T19:51:54.507908+0000","last_fullsized":"2024-03-11T19:51:54.507908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709765+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:51:56.709765+0000","last_peered":"2024-03-11T19:51:56.709765+0000","last_clean":"2024-03-11T19:51:56.709765+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:51:56.709765+0000","last_undegraded":"2024-03-11T19:51:56.709765+0000","last_fullsized":"2024-03-11T19:51:56.709765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617134+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:51:54.617134+0000","last_peered":"2024-03-11T19:51:54.617134+0000","last_clean":"2024-03-11T19:51:54.617134+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:51:54.617134+0000","last_undegraded":"2024-03-11T19:51:54.617134+0000","last_fullsized":"2024-03-11T19:51:54.617134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617063+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:51:54.617063+0000","last_peered":"2024-03-11T19:51:54.617063+0000","last_clean":"2024-03-11T19:51:54.617063+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:51:54.617063+0000","last_undegraded":"2024-03-11T19:51:54.617063+0000","last_fullsized":"2024-03-11T19:51:54.617063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523700+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:51:55.523700+0000","last_peered":"2024-03-11T19:51:55.523700+0000","last_clean":"2024-03-11T19:51:55.523700+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:51:55.523700+0000","last_undegraded":"2024-03-11T19:51:55.523700+0000","last_fullsized":"2024-03-11T19:51:55.523700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.326624+0000","last_change":"2024-03-11T19:49:55.218937+0000","last_active":"2024-03-11T19:51:54.326624+0000","last_peered":"2024-03-11T19:51:54.326624+0000","last_clean":"2024-03-11T19:51:54.326624+0000","last_became_active":"2024-03-11T19:49:55.218621+0000","last_became_peered":"2024-03-11T19:49:55.218621+0000","last_unstale":"2024-03-11T19:51:54.326624+0000","last_undegraded":"2024-03-11T19:51:54.326624+0000","last_fullsized":"2024-03-11T19:51:54.326624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:49:56.928038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709691+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:51:56.709691+0000","last_peered":"2024-03-11T19:51:56.709691+0000","last_clean":"2024-03-11T19:51:56.709691+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:51:56.709691+0000","last_undegraded":"2024-03-11T19:51:56.709691+0000","last_fullsized":"2024-03-11T19:51:56.709691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.617578+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:51:54.617578+0000","last_peered":"2024-03-11T19:51:54.617578+0000","last_clean":"2024-03-11T19:51:54.617578+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:51:54.617578+0000","last_undegraded":"2024-03-11T19:51:54.617578+0000","last_fullsized":"2024-03-11T19:51:54.617578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527102+0000","last_change":"2024-03-11T19:50:55.488762+0000","last_active":"2024-03-11T19:51:56.527102+0000","last_peered":"2024-03-11T19:51:56.527102+0000","last_clean":"2024-03-11T19:51:56.527102+0000","last_became_active":"2024-03-11T19:50:55.488324+0000","last_became_peered":"2024-03-11T19:50:55.488324+0000","last_unstale":"2024-03-11T19:51:56.527102+0000","last_undegraded":"2024-03-11T19:51:56.527102+0000","last_fullsized":"2024-03-11T19:51:56.527102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:36.522393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:50:55.472600+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:50:55.472600+0000","last_peered":"2024-03-11T19:50:55.472600+0000","last_clean":"2024-03-11T19:50:55.472600+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:50:55.472600+0000","last_undegraded":"2024-03-11T19:50:55.472600+0000","last_fullsized":"2024-03-11T19:50:55.472600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525884+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:51:56.525884+0000","last_peered":"2024-03-11T19:51:56.525884+0000","last_clean":"2024-03-11T19:51:56.525884+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:51:56.525884+0000","last_undegraded":"2024-03-11T19:51:56.525884+0000","last_fullsized":"2024-03-11T19:51:56.525884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709202+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:51:56.709202+0000","last_peered":"2024-03-11T19:51:56.709202+0000","last_clean":"2024-03-11T19:51:56.709202+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:51:56.709202+0000","last_undegraded":"2024-03-11T19:51:56.709202+0000","last_fullsized":"2024-03-11T19:51:56.709202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.325515+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:51:54.325515+0000","last_peered":"2024-03-11T19:51:54.325515+0000","last_clean":"2024-03-11T19:51:54.325515+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:51:54.325515+0000","last_undegraded":"2024-03-11T19:51:54.325515+0000","last_fullsized":"2024-03-11T19:51:54.325515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709904+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:51:56.709904+0000","last_peered":"2024-03-11T19:51:56.709904+0000","last_clean":"2024-03-11T19:51:56.709904+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:51:56.709904+0000","last_undegraded":"2024-03-11T19:51:56.709904+0000","last_fullsized":"2024-03-11T19:51:56.709904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":457,"ondisk_log_size":457,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66557,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":543,"num_read_kb":346,"num_write":171,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":199953,"data_compressed":4842,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":537362,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1187840,"data_stored":1191260,"data_compressed":67061,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518662,"data_compressed":23640,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":518560,"data_compressed":23640,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":1105458,"data_compressed":56469,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":504414,"data_compressed":20653,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":585374,"data_compressed":28246,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":524716,"data_compressed":26632,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":576388,"data_compressed":25254,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532946,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":510750,"data_compressed":20653,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":1100655,"data_compressed":56469,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62136,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62640,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:51:59.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:51:59.858 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:01.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:01 smithi107 ceph-mon[31712]: pgmap v388: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-11T19:52:01.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:01 smithi122 ceph-mon[36397]: pgmap v388: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-11T19:52:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:01 smithi145 ceph-mon[36730]: pgmap v388: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.126+0000 7f8513c00700 1 -- 172.21.15.107:0/119963207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c1028d0 msgr2=0x7f850c106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.126+0000 7f8513c00700 1 --2- 172.21.15.107:0/119963207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c1028d0 0x7f850c106910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0098f0 tx=0x7f84fc009c00 comp rx=0 tx=0).stop 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 -- 172.21.15.107:0/119963207 shutdown_connections 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 --2- 172.21.15.107:0/119963207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c10ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 --2- 172.21.15.107:0/119963207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c101f20 0x7f850c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 --2- 172.21.15.107:0/119963207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c1028d0 0x7f850c106910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 -- 172.21.15.107:0/119963207 >> 172.21.15.107:0/119963207 conn(0x7f850c078dd0 msgr2=0x7f850c0791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 -- 172.21.15.107:0/119963207 shutdown_connections 2024-03-11T19:52:02.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.127+0000 7f8513c00700 1 -- 172.21.15.107:0/119963207 wait complete. 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 Processor -- start 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 -- start start 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 0x7f850c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:02.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1a6660 con 0x7f850c106ff0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1a67d0 con 0x7f850c1028d0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.128+0000 7f8513c00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1a6990 con 0x7f850c101f20 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511200700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37664/0 (socket says 172.21.15.107:37664) 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511200700 1 -- 172.21.15.107:0/307417736 learned_addr learned my addr 172.21.15.107:0/307417736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 0x7f850c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 -- 172.21.15.107:0/307417736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 msgr2=0x7f850c073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 -- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 msgr2=0x7f850c0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 -- 172.21.15.107:0/307417736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009d00 con 0x7f850c1028d0 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511200700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:02.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8511c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.129+0000 7f8503e00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 0x7f850c0744b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009230 tx=0x7f84fc00fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.130+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01c070 con 0x7f850c1028d0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.130+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc009520 con 0x7f850c1028d0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.130+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84fc00b560 con 0x7f850c1028d0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.130+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc003bf0 con 0x7f850c1028d0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.130+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1a6ea0 con 0x7f850c1028d0 2024-03-11T19:52:02.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.131+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c10a310 con 0x7f850c1028d0 2024-03-11T19:52:02.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.133+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f84fc003d50 con 0x7f850c1028d0 2024-03-11T19:52:02.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.134+0000 7f850a000700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 0x7f84ec0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:02.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.134+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84fc0ce300 con 0x7f850c1028d0 2024-03-11T19:52:02.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.134+0000 7f8511200700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 0x7f84ec0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:02.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.135+0000 7f8511200700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 0x7f84ec0a3f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011550 tx=0x7f84f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:02.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.137+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84fc097190 con 0x7f850c1028d0 2024-03-11T19:52:02.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.351+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f850c111d30 con 0x7f84ec0a1ac0 2024-03-11T19:52:02.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.381+0000 7f850a000700 1 -- 172.21.15.107:0/307417736 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340885 (secure 0 0 0) 0x7f850c111d30 con 0x7f84ec0a1ac0 2024-03-11T19:52:02.392 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 msgr2=0x7f84ec0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 0x7f84ec0a3f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84f4011550 tx=0x7f84f4009260 comp rx=0 tx=0).stop 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 msgr2=0x7f850c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 0x7f850c0744b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009230 tx=0x7f84fc00fa90 comp rx=0 tx=0).stop 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 shutdown_connections 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f84ec0a1ac0 0x7f84ec0a3f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c106ff0 0x7f850c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f850c101f20 0x7f850c073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 --2- 172.21.15.107:0/307417736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f850c1028d0 0x7f850c0744b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 >> 172.21.15.107:0/307417736 conn(0x7f850c078dd0 msgr2=0x7f850c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 shutdown_connections 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:02.387+0000 7f8513c00700 1 -- 172.21.15.107:0/307417736 wait complete. 2024-03-11T19:52:02.396 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:52:03.321 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-03-11T19:52:00.377199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":695,"num_read_kb":471,"num_write":355,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182000,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353604,"kb_used_data":12668,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588220,"statfs":{"total":1151940427776,"available":1151578337280,"internally_reserved":0,"allocated":12972032,"data_stored":8358148,"data_compressed":455873,"data_compressed_allocated":622592,"data_compressed_original":2744320,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":23,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002313"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709765+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:51:56.709765+0000","last_peered":"2024-03-11T19:51:56.709765+0000","last_clean":"2024-03-11T19:51:56.709765+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:51:56.709765+0000","last_undegraded":"2024-03-11T19:51:56.709765+0000","last_fullsized":"2024-03-11T19:51:56.709765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.525207+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:51:55.525207+0000","last_peered":"2024-03-11T19:51:55.525207+0000","last_clean":"2024-03-11T19:51:55.525207+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:51:55.525207+0000","last_undegraded":"2024-03-11T19:51:55.525207+0000","last_fullsized":"2024-03-11T19:51:55.525207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.524067+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:51:55.524067+0000","last_peered":"2024-03-11T19:51:55.524067+0000","last_clean":"2024-03-11T19:51:55.524067+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:51:55.524067+0000","last_undegraded":"2024-03-11T19:51:55.524067+0000","last_fullsized":"2024-03-11T19:51:55.524067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523239+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:51:56.523239+0000","last_peered":"2024-03-11T19:51:56.523239+0000","last_clean":"2024-03-11T19:51:56.523239+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:51:56.523239+0000","last_undegraded":"2024-03-11T19:51:56.523239+0000","last_fullsized":"2024-03-11T19:51:56.523239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522831+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:51:56.522831+0000","last_peered":"2024-03-11T19:51:56.522831+0000","last_clean":"2024-03-11T19:51:56.522831+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:51:56.522831+0000","last_undegraded":"2024-03-11T19:51:56.522831+0000","last_fullsized":"2024-03-11T19:51:56.522831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532313+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:51:56.532313+0000","last_peered":"2024-03-11T19:51:56.532313+0000","last_clean":"2024-03-11T19:51:56.532313+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:51:56.532313+0000","last_undegraded":"2024-03-11T19:51:56.532313+0000","last_fullsized":"2024-03-11T19:51:56.532313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709889+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:51:56.709889+0000","last_peered":"2024-03-11T19:51:56.709889+0000","last_clean":"2024-03-11T19:51:56.709889+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:51:56.709889+0000","last_undegraded":"2024-03-11T19:51:56.709889+0000","last_fullsized":"2024-03-11T19:51:56.709889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.524741+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:51:56.524741+0000","last_peered":"2024-03-11T19:51:56.524741+0000","last_clean":"2024-03-11T19:51:56.524741+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:51:56.524741+0000","last_undegraded":"2024-03-11T19:51:56.524741+0000","last_fullsized":"2024-03-11T19:51:56.524741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709398+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:51:56.709398+0000","last_peered":"2024-03-11T19:51:56.709398+0000","last_clean":"2024-03-11T19:51:56.709398+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:51:56.709398+0000","last_undegraded":"2024-03-11T19:51:56.709398+0000","last_fullsized":"2024-03-11T19:51:56.709398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532578+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:51:56.532578+0000","last_peered":"2024-03-11T19:51:56.532578+0000","last_clean":"2024-03-11T19:51:56.532578+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:51:56.532578+0000","last_undegraded":"2024-03-11T19:51:56.532578+0000","last_fullsized":"2024-03-11T19:51:56.532578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709875+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:51:56.709875+0000","last_peered":"2024-03-11T19:51:56.709875+0000","last_clean":"2024-03-11T19:51:56.709875+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:51:56.709875+0000","last_undegraded":"2024-03-11T19:51:56.709875+0000","last_fullsized":"2024-03-11T19:51:56.709875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523155+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:51:56.523155+0000","last_peered":"2024-03-11T19:51:56.523155+0000","last_clean":"2024-03-11T19:51:56.523155+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:51:56.523155+0000","last_undegraded":"2024-03-11T19:51:56.523155+0000","last_fullsized":"2024-03-11T19:51:56.523155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716871+0000","last_change":"2024-03-11T19:51:56.716871+0000","last_active":"2024-03-11T19:51:56.716871+0000","last_peered":"2024-03-11T19:51:56.716871+0000","last_clean":"2024-03-11T19:51:56.716871+0000","last_became_active":"2024-03-11T19:51:56.715370+0000","last_became_peered":"2024-03-11T19:51:56.715370+0000","last_unstale":"2024-03-11T19:51:56.716871+0000","last_undegraded":"2024-03-11T19:51:56.716871+0000","last_fullsized":"2024-03-11T19:51:56.716871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:13:56.734860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523062+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:51:55.523062+0000","last_peered":"2024-03-11T19:51:55.523062+0000","last_clean":"2024-03-11T19:51:55.523062+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:51:55.523062+0000","last_undegraded":"2024-03-11T19:51:55.523062+0000","last_fullsized":"2024-03-11T19:51:55.523062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.522612+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:51:55.522612+0000","last_peered":"2024-03-11T19:51:55.522612+0000","last_clean":"2024-03-11T19:51:55.522612+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:51:55.522612+0000","last_undegraded":"2024-03-11T19:51:55.522612+0000","last_fullsized":"2024-03-11T19:51:55.522612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709009+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:51:56.709009+0000","last_peered":"2024-03-11T19:51:56.709009+0000","last_clean":"2024-03-11T19:51:56.709009+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:51:56.709009+0000","last_undegraded":"2024-03-11T19:51:56.709009+0000","last_fullsized":"2024-03-11T19:51:56.709009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709723+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:51:56.709723+0000","last_peered":"2024-03-11T19:51:56.709723+0000","last_clean":"2024-03-11T19:51:56.709723+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:51:56.709723+0000","last_undegraded":"2024-03-11T19:51:56.709723+0000","last_fullsized":"2024-03-11T19:51:56.709723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514991+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:51:54.514991+0000","last_peered":"2024-03-11T19:51:54.514991+0000","last_clean":"2024-03-11T19:51:54.514991+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:51:54.514991+0000","last_undegraded":"2024-03-11T19:51:54.514991+0000","last_fullsized":"2024-03-11T19:51:54.514991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709909+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:51:56.709909+0000","last_peered":"2024-03-11T19:51:56.709909+0000","last_clean":"2024-03-11T19:51:56.709909+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:51:56.709909+0000","last_undegraded":"2024-03-11T19:51:56.709909+0000","last_fullsized":"2024-03-11T19:51:56.709909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708991+0000","last_change":"2024-03-11T19:50:55.487486+0000","last_active":"2024-03-11T19:51:56.708991+0000","last_peered":"2024-03-11T19:51:56.708991+0000","last_clean":"2024-03-11T19:51:56.708991+0000","last_became_active":"2024-03-11T19:50:55.486998+0000","last_became_peered":"2024-03-11T19:50:55.486998+0000","last_unstale":"2024-03-11T19:51:56.708991+0000","last_undegraded":"2024-03-11T19:51:56.708991+0000","last_fullsized":"2024-03-11T19:51:56.708991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:39.904812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.515063+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:51:54.515063+0000","last_peered":"2024-03-11T19:51:54.515063+0000","last_clean":"2024-03-11T19:51:54.515063+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:51:54.515063+0000","last_undegraded":"2024-03-11T19:51:54.515063+0000","last_fullsized":"2024-03-11T19:51:54.515063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526502+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:51:56.526502+0000","last_peered":"2024-03-11T19:51:56.526502+0000","last_clean":"2024-03-11T19:51:56.526502+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:51:56.526502+0000","last_undegraded":"2024-03-11T19:51:56.526502+0000","last_fullsized":"2024-03-11T19:51:56.526502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522659+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:51:56.522659+0000","last_peered":"2024-03-11T19:51:56.522659+0000","last_clean":"2024-03-11T19:51:56.522659+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:51:56.522659+0000","last_undegraded":"2024-03-11T19:51:56.522659+0000","last_fullsized":"2024-03-11T19:51:56.522659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527995+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:51:56.527995+0000","last_peered":"2024-03-11T19:51:56.527995+0000","last_clean":"2024-03-11T19:51:56.527995+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:51:56.527995+0000","last_undegraded":"2024-03-11T19:51:56.527995+0000","last_fullsized":"2024-03-11T19:51:56.527995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.528111+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:51:55.528111+0000","last_peered":"2024-03-11T19:51:55.528111+0000","last_clean":"2024-03-11T19:51:55.528111+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:51:55.528111+0000","last_undegraded":"2024-03-11T19:51:55.528111+0000","last_fullsized":"2024-03-11T19:51:55.528111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522810+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:51:56.522810+0000","last_peered":"2024-03-11T19:51:56.522810+0000","last_clean":"2024-03-11T19:51:56.522810+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:51:56.522810+0000","last_undegraded":"2024-03-11T19:51:56.522810+0000","last_fullsized":"2024-03-11T19:51:56.522810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709813+0000","last_change":"2024-03-11T19:50:55.480971+0000","last_active":"2024-03-11T19:51:56.709813+0000","last_peered":"2024-03-11T19:51:56.709813+0000","last_clean":"2024-03-11T19:51:56.709813+0000","last_became_active":"2024-03-11T19:50:55.474612+0000","last_became_peered":"2024-03-11T19:50:55.474612+0000","last_unstale":"2024-03-11T19:51:56.709813+0000","last_undegraded":"2024-03-11T19:51:56.709813+0000","last_fullsized":"2024-03-11T19:51:56.709813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:59:47.829059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522632+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:51:56.522632+0000","last_peered":"2024-03-11T19:51:56.522632+0000","last_clean":"2024-03-11T19:51:56.522632+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:51:56.522632+0000","last_undegraded":"2024-03-11T19:51:56.522632+0000","last_fullsized":"2024-03-11T19:51:56.522632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522758+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:51:56.522758+0000","last_peered":"2024-03-11T19:51:56.522758+0000","last_clean":"2024-03-11T19:51:56.522758+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:51:56.522758+0000","last_undegraded":"2024-03-11T19:51:56.522758+0000","last_fullsized":"2024-03-11T19:51:56.522758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596489+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:51:56.596489+0000","last_peered":"2024-03-11T19:51:56.596489+0000","last_clean":"2024-03-11T19:51:56.596489+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:51:56.596489+0000","last_undegraded":"2024-03-11T19:51:56.596489+0000","last_fullsized":"2024-03-11T19:51:56.596489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709721+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:51:56.709721+0000","last_peered":"2024-03-11T19:51:56.709721+0000","last_clean":"2024-03-11T19:51:56.709721+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:51:56.709721+0000","last_undegraded":"2024-03-11T19:51:56.709721+0000","last_fullsized":"2024-03-11T19:51:56.709721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":145,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596870+0000","last_change":"2024-03-11T19:50:55.509345+0000","last_active":"2024-03-11T19:51:56.596870+0000","last_peered":"2024-03-11T19:51:56.596870+0000","last_clean":"2024-03-11T19:51:56.596870+0000","last_became_active":"2024-03-11T19:50:55.505792+0000","last_became_peered":"2024-03-11T19:50:55.505792+0000","last_unstale":"2024-03-11T19:51:56.596870+0000","last_undegraded":"2024-03-11T19:51:56.596870+0000","last_fullsized":"2024-03-11T19:51:56.596870+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:01:20.682572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522735+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:51:56.522735+0000","last_peered":"2024-03-11T19:51:56.522735+0000","last_clean":"2024-03-11T19:51:56.522735+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:51:56.522735+0000","last_undegraded":"2024-03-11T19:51:56.522735+0000","last_fullsized":"2024-03-11T19:51:56.522735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709487+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:51:56.709487+0000","last_peered":"2024-03-11T19:51:56.709487+0000","last_clean":"2024-03-11T19:51:56.709487+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:51:56.709487+0000","last_undegraded":"2024-03-11T19:51:56.709487+0000","last_fullsized":"2024-03-11T19:51:56.709487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532197+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:51:56.532197+0000","last_peered":"2024-03-11T19:51:56.532197+0000","last_clean":"2024-03-11T19:51:56.532197+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:51:56.532197+0000","last_undegraded":"2024-03-11T19:51:56.532197+0000","last_fullsized":"2024-03-11T19:51:56.532197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596519+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:51:56.596519+0000","last_peered":"2024-03-11T19:51:56.596519+0000","last_clean":"2024-03-11T19:51:56.596519+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:51:56.596519+0000","last_undegraded":"2024-03-11T19:51:56.596519+0000","last_fullsized":"2024-03-11T19:51:56.596519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.529353+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:51:56.529353+0000","last_peered":"2024-03-11T19:51:56.529353+0000","last_clean":"2024-03-11T19:51:56.529353+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:51:56.529353+0000","last_undegraded":"2024-03-11T19:51:56.529353+0000","last_fullsized":"2024-03-11T19:51:56.529353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":2,"reported_epoch":93,"state":"peering","last_fresh":"2024-03-11T19:51:54.325327+0000","last_change":"2024-03-11T19:51:55.398160+0000","last_active":"2024-03-11T19:51:55.398160+0000","last_peered":"2024-03-11T19:51:54.325327+0000","last_clean":"2024-03-11T19:51:54.325327+0000","last_became_active":"2024-03-11T19:49:55.236259+0000","last_became_peered":"2024-03-11T19:49:55.236259+0000","last_unstale":"2024-03-11T19:51:54.325327+0000","last_undegraded":"2024-03-11T19:51:54.325327+0000","last_fullsized":"2024-03-11T19:51:54.325327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.524925+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:51:55.524925+0000","last_peered":"2024-03-11T19:51:55.524925+0000","last_clean":"2024-03-11T19:51:55.524925+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:51:55.524925+0000","last_undegraded":"2024-03-11T19:51:55.524925+0000","last_fullsized":"2024-03-11T19:51:55.524925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596062+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:51:56.596062+0000","last_peered":"2024-03-11T19:51:56.596062+0000","last_clean":"2024-03-11T19:51:56.596062+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:51:56.596062+0000","last_undegraded":"2024-03-11T19:51:56.596062+0000","last_fullsized":"2024-03-11T19:51:56.596062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513839+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:51:54.513839+0000","last_peered":"2024-03-11T19:51:54.513839+0000","last_clean":"2024-03-11T19:51:54.513839+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:51:54.513839+0000","last_undegraded":"2024-03-11T19:51:54.513839+0000","last_fullsized":"2024-03-11T19:51:54.513839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525570+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:51:56.525570+0000","last_peered":"2024-03-11T19:51:56.525570+0000","last_clean":"2024-03-11T19:51:56.525570+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:51:56.525570+0000","last_undegraded":"2024-03-11T19:51:56.525570+0000","last_fullsized":"2024-03-11T19:51:56.525570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.594032+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:51:56.594032+0000","last_peered":"2024-03-11T19:51:56.594032+0000","last_clean":"2024-03-11T19:51:56.594032+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:51:56.594032+0000","last_undegraded":"2024-03-11T19:51:56.594032+0000","last_fullsized":"2024-03-11T19:51:56.594032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709914+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:51:56.709914+0000","last_peered":"2024-03-11T19:51:56.709914+0000","last_clean":"2024-03-11T19:51:56.709914+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:51:56.709914+0000","last_undegraded":"2024-03-11T19:51:56.709914+0000","last_fullsized":"2024-03-11T19:51:56.709914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513462+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:51:54.513462+0000","last_peered":"2024-03-11T19:51:54.513462+0000","last_clean":"2024-03-11T19:51:54.513462+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:51:54.513462+0000","last_undegraded":"2024-03-11T19:51:54.513462+0000","last_fullsized":"2024-03-11T19:51:54.513462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709237+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:51:56.709237+0000","last_peered":"2024-03-11T19:51:56.709237+0000","last_clean":"2024-03-11T19:51:56.709237+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:51:56.709237+0000","last_undegraded":"2024-03-11T19:51:56.709237+0000","last_fullsized":"2024-03-11T19:51:56.709237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.547076+0000","last_change":"2024-03-11T19:51:55.547076+0000","last_active":"2024-03-11T19:51:55.547076+0000","last_peered":"2024-03-11T19:51:55.547076+0000","last_clean":"2024-03-11T19:51:55.547076+0000","last_became_active":"2024-03-11T19:51:55.546410+0000","last_became_peered":"2024-03-11T19:51:55.546410+0000","last_unstale":"2024-03-11T19:51:55.547076+0000","last_undegraded":"2024-03-11T19:51:55.547076+0000","last_fullsized":"2024-03-11T19:51:55.547076+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:43:21.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522940+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:51:56.522940+0000","last_peered":"2024-03-11T19:51:56.522940+0000","last_clean":"2024-03-11T19:51:56.522940+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:51:56.522940+0000","last_undegraded":"2024-03-11T19:51:56.522940+0000","last_fullsized":"2024-03-11T19:51:56.522940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596354+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:51:56.596354+0000","last_peered":"2024-03-11T19:51:56.596354+0000","last_clean":"2024-03-11T19:51:56.596354+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:51:56.596354+0000","last_undegraded":"2024-03-11T19:51:56.596354+0000","last_fullsized":"2024-03-11T19:51:56.596354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532035+0000","last_change":"2024-03-11T19:50:55.475760+0000","last_active":"2024-03-11T19:51:56.532035+0000","last_peered":"2024-03-11T19:51:56.532035+0000","last_clean":"2024-03-11T19:51:56.532035+0000","last_became_active":"2024-03-11T19:50:55.475170+0000","last_became_peered":"2024-03-11T19:50:55.475170+0000","last_unstale":"2024-03-11T19:51:56.532035+0000","last_undegraded":"2024-03-11T19:51:56.532035+0000","last_fullsized":"2024-03-11T19:51:56.532035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:58.278212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":252,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:57.597589+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:51:57.597589+0000","last_peered":"2024-03-11T19:51:57.597589+0000","last_clean":"2024-03-11T19:51:57.597589+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:51:57.597589+0000","last_undegraded":"2024-03-11T19:51:57.597589+0000","last_fullsized":"2024-03-11T19:51:57.597589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":184,"num_read_kb":153,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523089+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:51:56.523089+0000","last_peered":"2024-03-11T19:51:56.523089+0000","last_clean":"2024-03-11T19:51:56.523089+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:51:56.523089+0000","last_undegraded":"2024-03-11T19:51:56.523089+0000","last_fullsized":"2024-03-11T19:51:56.523089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596828+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:51:56.596828+0000","last_peered":"2024-03-11T19:51:56.596828+0000","last_clean":"2024-03-11T19:51:56.596828+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:51:56.596828+0000","last_undegraded":"2024-03-11T19:51:56.596828+0000","last_fullsized":"2024-03-11T19:51:56.596828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709030+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:51:56.709030+0000","last_peered":"2024-03-11T19:51:56.709030+0000","last_clean":"2024-03-11T19:51:56.709030+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:51:56.709030+0000","last_undegraded":"2024-03-11T19:51:56.709030+0000","last_fullsized":"2024-03-11T19:51:56.709030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530696+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:51:56.530696+0000","last_peered":"2024-03-11T19:51:56.530696+0000","last_clean":"2024-03-11T19:51:56.530696+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:51:56.530696+0000","last_undegraded":"2024-03-11T19:51:56.530696+0000","last_fullsized":"2024-03-11T19:51:56.530696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709442+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:51:56.709442+0000","last_peered":"2024-03-11T19:51:56.709442+0000","last_clean":"2024-03-11T19:51:56.709442+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:51:56.709442+0000","last_undegraded":"2024-03-11T19:51:56.709442+0000","last_fullsized":"2024-03-11T19:51:56.709442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596343+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:51:56.596343+0000","last_peered":"2024-03-11T19:51:56.596343+0000","last_clean":"2024-03-11T19:51:56.596343+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:51:56.596343+0000","last_undegraded":"2024-03-11T19:51:56.596343+0000","last_fullsized":"2024-03-11T19:51:56.596343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710085+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:51:56.710085+0000","last_peered":"2024-03-11T19:51:56.710085+0000","last_clean":"2024-03-11T19:51:56.710085+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:51:56.710085+0000","last_undegraded":"2024-03-11T19:51:56.710085+0000","last_fullsized":"2024-03-11T19:51:56.710085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514760+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:51:54.514760+0000","last_peered":"2024-03-11T19:51:54.514760+0000","last_clean":"2024-03-11T19:51:54.514760+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:51:54.514760+0000","last_undegraded":"2024-03-11T19:51:54.514760+0000","last_fullsized":"2024-03-11T19:51:54.514760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":134,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:58.640851+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:51:58.640851+0000","last_peered":"2024-03-11T19:51:58.640851+0000","last_clean":"2024-03-11T19:51:58.640851+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:51:58.640851+0000","last_undegraded":"2024-03-11T19:51:58.640851+0000","last_fullsized":"2024-03-11T19:51:58.640851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708887+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:51:56.708887+0000","last_peered":"2024-03-11T19:51:56.708887+0000","last_clean":"2024-03-11T19:51:56.708887+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:51:56.708887+0000","last_undegraded":"2024-03-11T19:51:56.708887+0000","last_fullsized":"2024-03-11T19:51:56.708887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531921+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:51:56.531921+0000","last_peered":"2024-03-11T19:51:56.531921+0000","last_clean":"2024-03-11T19:51:56.531921+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:51:56.531921+0000","last_undegraded":"2024-03-11T19:51:56.531921+0000","last_fullsized":"2024-03-11T19:51:56.531921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522544+0000","last_change":"2024-03-11T19:50:55.807772+0000","last_active":"2024-03-11T19:51:56.522544+0000","last_peered":"2024-03-11T19:51:56.522544+0000","last_clean":"2024-03-11T19:51:56.522544+0000","last_became_active":"2024-03-11T19:50:55.794019+0000","last_became_peered":"2024-03-11T19:50:55.794019+0000","last_unstale":"2024-03-11T19:51:56.522544+0000","last_undegraded":"2024-03-11T19:51:56.522544+0000","last_fullsized":"2024-03-11T19:51:56.522544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:00.170150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596163+0000","last_change":"2024-03-11T19:50:55.793868+0000","last_active":"2024-03-11T19:51:56.596163+0000","last_peered":"2024-03-11T19:51:56.596163+0000","last_clean":"2024-03-11T19:51:56.596163+0000","last_became_active":"2024-03-11T19:50:55.793299+0000","last_became_peered":"2024-03-11T19:50:55.793299+0000","last_unstale":"2024-03-11T19:51:56.596163+0000","last_undegraded":"2024-03-11T19:51:56.596163+0000","last_fullsized":"2024-03-11T19:51:56.596163+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:40:42.039449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596033+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:51:56.596033+0000","last_peered":"2024-03-11T19:51:56.596033+0000","last_clean":"2024-03-11T19:51:56.596033+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:51:56.596033+0000","last_undegraded":"2024-03-11T19:51:56.596033+0000","last_fullsized":"2024-03-11T19:51:56.596033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523361+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:51:56.523361+0000","last_peered":"2024-03-11T19:51:56.523361+0000","last_clean":"2024-03-11T19:51:56.523361+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:51:56.523361+0000","last_undegraded":"2024-03-11T19:51:56.523361+0000","last_fullsized":"2024-03-11T19:51:56.523361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709559+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:51:56.709559+0000","last_peered":"2024-03-11T19:51:56.709559+0000","last_clean":"2024-03-11T19:51:56.709559+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:51:56.709559+0000","last_undegraded":"2024-03-11T19:51:56.709559+0000","last_fullsized":"2024-03-11T19:51:56.709559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522734+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:51:56.522734+0000","last_peered":"2024-03-11T19:51:56.522734+0000","last_clean":"2024-03-11T19:51:56.522734+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:51:56.522734+0000","last_undegraded":"2024-03-11T19:51:56.522734+0000","last_fullsized":"2024-03-11T19:51:56.522734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.513579+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:51:54.513579+0000","last_peered":"2024-03-11T19:51:54.513579+0000","last_clean":"2024-03-11T19:51:54.513579+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:51:54.513579+0000","last_undegraded":"2024-03-11T19:51:54.513579+0000","last_fullsized":"2024-03-11T19:51:54.513579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532533+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:51:56.532533+0000","last_peered":"2024-03-11T19:51:56.532533+0000","last_clean":"2024-03-11T19:51:56.532533+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:51:56.532533+0000","last_undegraded":"2024-03-11T19:51:56.532533+0000","last_fullsized":"2024-03-11T19:51:56.532533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531395+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:51:56.531395+0000","last_peered":"2024-03-11T19:51:56.531395+0000","last_clean":"2024-03-11T19:51:56.531395+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:51:56.531395+0000","last_undegraded":"2024-03-11T19:51:56.531395+0000","last_fullsized":"2024-03-11T19:51:56.531395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709519+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:51:56.709519+0000","last_peered":"2024-03-11T19:51:56.709519+0000","last_clean":"2024-03-11T19:51:56.709519+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:51:56.709519+0000","last_undegraded":"2024-03-11T19:51:56.709519+0000","last_fullsized":"2024-03-11T19:51:56.709519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532656+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:51:56.532656+0000","last_peered":"2024-03-11T19:51:56.532656+0000","last_clean":"2024-03-11T19:51:56.532656+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:51:56.532656+0000","last_undegraded":"2024-03-11T19:51:56.532656+0000","last_fullsized":"2024-03-11T19:51:56.532656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532300+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:51:56.532300+0000","last_peered":"2024-03-11T19:51:56.532300+0000","last_clean":"2024-03-11T19:51:56.532300+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:51:56.532300+0000","last_undegraded":"2024-03-11T19:51:56.532300+0000","last_fullsized":"2024-03-11T19:51:56.532300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596553+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:51:56.596553+0000","last_peered":"2024-03-11T19:51:56.596553+0000","last_clean":"2024-03-11T19:51:56.596553+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:51:56.596553+0000","last_undegraded":"2024-03-11T19:51:56.596553+0000","last_fullsized":"2024-03-11T19:51:56.596553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":130,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:58.639702+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:51:58.639702+0000","last_peered":"2024-03-11T19:51:58.639702+0000","last_clean":"2024-03-11T19:51:58.639702+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:51:58.639702+0000","last_undegraded":"2024-03-11T19:51:58.639702+0000","last_fullsized":"2024-03-11T19:51:58.639702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709555+0000","last_change":"2024-03-11T19:50:55.488155+0000","last_active":"2024-03-11T19:51:56.709555+0000","last_peered":"2024-03-11T19:51:56.709555+0000","last_clean":"2024-03-11T19:51:56.709555+0000","last_became_active":"2024-03-11T19:50:55.477254+0000","last_became_peered":"2024-03-11T19:50:55.477254+0000","last_unstale":"2024-03-11T19:51:56.709555+0000","last_undegraded":"2024-03-11T19:51:56.709555+0000","last_fullsized":"2024-03-11T19:51:56.709555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:05.827200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527168+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:51:56.527168+0000","last_peered":"2024-03-11T19:51:56.527168+0000","last_clean":"2024-03-11T19:51:56.527168+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:51:56.527168+0000","last_undegraded":"2024-03-11T19:51:56.527168+0000","last_fullsized":"2024-03-11T19:51:56.527168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522985+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:51:56.522985+0000","last_peered":"2024-03-11T19:51:56.522985+0000","last_clean":"2024-03-11T19:51:56.522985+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:51:56.522985+0000","last_undegraded":"2024-03-11T19:51:56.522985+0000","last_fullsized":"2024-03-11T19:51:56.522985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708751+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:51:56.708751+0000","last_peered":"2024-03-11T19:51:56.708751+0000","last_clean":"2024-03-11T19:51:56.708751+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:51:56.708751+0000","last_undegraded":"2024-03-11T19:51:56.708751+0000","last_fullsized":"2024-03-11T19:51:56.708751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596781+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:51:56.596781+0000","last_peered":"2024-03-11T19:51:56.596781+0000","last_clean":"2024-03-11T19:51:56.596781+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:51:56.596781+0000","last_undegraded":"2024-03-11T19:51:56.596781+0000","last_fullsized":"2024-03-11T19:51:56.596781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.528022+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:51:56.528022+0000","last_peered":"2024-03-11T19:51:56.528022+0000","last_clean":"2024-03-11T19:51:56.528022+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:51:56.528022+0000","last_undegraded":"2024-03-11T19:51:56.528022+0000","last_fullsized":"2024-03-11T19:51:56.528022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708807+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:51:56.708807+0000","last_peered":"2024-03-11T19:51:56.708807+0000","last_clean":"2024-03-11T19:51:56.708807+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:51:56.708807+0000","last_undegraded":"2024-03-11T19:51:56.708807+0000","last_fullsized":"2024-03-11T19:51:56.708807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709301+0000","last_change":"2024-03-11T19:50:55.474853+0000","last_active":"2024-03-11T19:51:56.709301+0000","last_peered":"2024-03-11T19:51:56.709301+0000","last_clean":"2024-03-11T19:51:56.709301+0000","last_became_active":"2024-03-11T19:50:55.474379+0000","last_became_peered":"2024-03-11T19:50:55.474379+0000","last_unstale":"2024-03-11T19:51:56.709301+0000","last_undegraded":"2024-03-11T19:51:56.709301+0000","last_fullsized":"2024-03-11T19:51:56.709301+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:43.691094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709662+0000","last_change":"2024-03-11T19:50:55.491848+0000","last_active":"2024-03-11T19:51:56.709662+0000","last_peered":"2024-03-11T19:51:56.709662+0000","last_clean":"2024-03-11T19:51:56.709662+0000","last_became_active":"2024-03-11T19:50:55.485786+0000","last_became_peered":"2024-03-11T19:50:55.485786+0000","last_unstale":"2024-03-11T19:51:56.709662+0000","last_undegraded":"2024-03-11T19:51:56.709662+0000","last_fullsized":"2024-03-11T19:51:56.709662+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:24:41.406176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532569+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:51:56.532569+0000","last_peered":"2024-03-11T19:51:56.532569+0000","last_clean":"2024-03-11T19:51:56.532569+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:51:56.532569+0000","last_undegraded":"2024-03-11T19:51:56.532569+0000","last_fullsized":"2024-03-11T19:51:56.532569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525078+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:51:56.525078+0000","last_peered":"2024-03-11T19:51:56.525078+0000","last_clean":"2024-03-11T19:51:56.525078+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:51:56.525078+0000","last_undegraded":"2024-03-11T19:51:56.525078+0000","last_fullsized":"2024-03-11T19:51:56.525078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514897+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:51:54.514897+0000","last_peered":"2024-03-11T19:51:54.514897+0000","last_clean":"2024-03-11T19:51:54.514897+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:51:54.514897+0000","last_undegraded":"2024-03-11T19:51:54.514897+0000","last_fullsized":"2024-03-11T19:51:54.514897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526601+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:51:56.526601+0000","last_peered":"2024-03-11T19:51:56.526601+0000","last_clean":"2024-03-11T19:51:56.526601+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:51:56.526601+0000","last_undegraded":"2024-03-11T19:51:56.526601+0000","last_fullsized":"2024-03-11T19:51:56.526601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708706+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:51:56.708706+0000","last_peered":"2024-03-11T19:51:56.708706+0000","last_clean":"2024-03-11T19:51:56.708706+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:51:56.708706+0000","last_undegraded":"2024-03-11T19:51:56.708706+0000","last_fullsized":"2024-03-11T19:51:56.708706+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523340+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:51:56.523340+0000","last_peered":"2024-03-11T19:51:56.523340+0000","last_clean":"2024-03-11T19:51:56.523340+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:51:56.523340+0000","last_undegraded":"2024-03-11T19:51:56.523340+0000","last_fullsized":"2024-03-11T19:51:56.523340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.525319+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:51:55.525319+0000","last_peered":"2024-03-11T19:51:55.525319+0000","last_clean":"2024-03-11T19:51:55.525319+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:51:55.525319+0000","last_undegraded":"2024-03-11T19:51:55.525319+0000","last_fullsized":"2024-03-11T19:51:55.525319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523190+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:51:56.523190+0000","last_peered":"2024-03-11T19:51:56.523190+0000","last_clean":"2024-03-11T19:51:56.523190+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:51:56.523190+0000","last_undegraded":"2024-03-11T19:51:56.523190+0000","last_fullsized":"2024-03-11T19:51:56.523190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709061+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:51:56.709061+0000","last_peered":"2024-03-11T19:51:56.709061+0000","last_clean":"2024-03-11T19:51:56.709061+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:51:56.709061+0000","last_undegraded":"2024-03-11T19:51:56.709061+0000","last_fullsized":"2024-03-11T19:51:56.709061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522822+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:51:56.522822+0000","last_peered":"2024-03-11T19:51:56.522822+0000","last_clean":"2024-03-11T19:51:56.522822+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:51:56.522822+0000","last_undegraded":"2024-03-11T19:51:56.522822+0000","last_fullsized":"2024-03-11T19:51:56.522822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-11T19:51:54.514956+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:51:54.514956+0000","last_peered":"2024-03-11T19:51:54.514956+0000","last_clean":"2024-03-11T19:51:54.514956+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:51:54.514956+0000","last_undegraded":"2024-03-11T19:51:54.514956+0000","last_fullsized":"2024-03-11T19:51:54.514956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525797+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:51:56.525797+0000","last_peered":"2024-03-11T19:51:56.525797+0000","last_clean":"2024-03-11T19:51:56.525797+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:51:56.525797+0000","last_undegraded":"2024-03-11T19:51:56.525797+0000","last_fullsized":"2024-03-11T19:51:56.525797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709514+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:51:56.709514+0000","last_peered":"2024-03-11T19:51:56.709514+0000","last_clean":"2024-03-11T19:51:56.709514+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:51:56.709514+0000","last_undegraded":"2024-03-11T19:51:56.709514+0000","last_fullsized":"2024-03-11T19:51:56.709514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523006+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:51:56.523006+0000","last_peered":"2024-03-11T19:51:56.523006+0000","last_clean":"2024-03-11T19:51:56.523006+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:51:56.523006+0000","last_undegraded":"2024-03-11T19:51:56.523006+0000","last_fullsized":"2024-03-11T19:51:56.523006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708841+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:51:56.708841+0000","last_peered":"2024-03-11T19:51:56.708841+0000","last_clean":"2024-03-11T19:51:56.708841+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:51:56.708841+0000","last_undegraded":"2024-03-11T19:51:56.708841+0000","last_fullsized":"2024-03-11T19:51:56.708841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525612+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:51:56.525612+0000","last_peered":"2024-03-11T19:51:56.525612+0000","last_clean":"2024-03-11T19:51:56.525612+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:51:56.525612+0000","last_undegraded":"2024-03-11T19:51:56.525612+0000","last_fullsized":"2024-03-11T19:51:56.525612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522988+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:51:56.522988+0000","last_peered":"2024-03-11T19:51:56.522988+0000","last_clean":"2024-03-11T19:51:56.522988+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:51:56.522988+0000","last_undegraded":"2024-03-11T19:51:56.522988+0000","last_fullsized":"2024-03-11T19:51:56.522988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522717+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:51:56.522717+0000","last_peered":"2024-03-11T19:51:56.522717+0000","last_clean":"2024-03-11T19:51:56.522717+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:51:56.522717+0000","last_undegraded":"2024-03-11T19:51:56.522717+0000","last_fullsized":"2024-03-11T19:51:56.522717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709536+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:51:56.709536+0000","last_peered":"2024-03-11T19:51:56.709536+0000","last_clean":"2024-03-11T19:51:56.709536+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:51:56.709536+0000","last_undegraded":"2024-03-11T19:51:56.709536+0000","last_fullsized":"2024-03-11T19:51:56.709536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708833+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:51:56.708833+0000","last_peered":"2024-03-11T19:51:56.708833+0000","last_clean":"2024-03-11T19:51:56.708833+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:51:56.708833+0000","last_undegraded":"2024-03-11T19:51:56.708833+0000","last_fullsized":"2024-03-11T19:51:56.708833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709374+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:51:56.709374+0000","last_peered":"2024-03-11T19:51:56.709374+0000","last_clean":"2024-03-11T19:51:56.709374+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:51:56.709374+0000","last_undegraded":"2024-03-11T19:51:56.709374+0000","last_fullsized":"2024-03-11T19:51:56.709374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710015+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:51:56.710015+0000","last_peered":"2024-03-11T19:51:56.710015+0000","last_clean":"2024-03-11T19:51:56.710015+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:51:56.710015+0000","last_undegraded":"2024-03-11T19:51:56.710015+0000","last_fullsized":"2024-03-11T19:51:56.710015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523185+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:51:55.523185+0000","last_peered":"2024-03-11T19:51:55.523185+0000","last_clean":"2024-03-11T19:51:55.523185+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:51:55.523185+0000","last_undegraded":"2024-03-11T19:51:55.523185+0000","last_fullsized":"2024-03-11T19:51:55.523185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596670+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:51:56.596670+0000","last_peered":"2024-03-11T19:51:56.596670+0000","last_clean":"2024-03-11T19:51:56.596670+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:51:56.596670+0000","last_undegraded":"2024-03-11T19:51:56.596670+0000","last_fullsized":"2024-03-11T19:51:56.596670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709620+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:51:56.709620+0000","last_peered":"2024-03-11T19:51:56.709620+0000","last_clean":"2024-03-11T19:51:56.709620+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:51:56.709620+0000","last_undegraded":"2024-03-11T19:51:56.709620+0000","last_fullsized":"2024-03-11T19:51:56.709620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.595922+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:51:56.595922+0000","last_peered":"2024-03-11T19:51:56.595922+0000","last_clean":"2024-03-11T19:51:56.595922+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:51:56.595922+0000","last_undegraded":"2024-03-11T19:51:56.595922+0000","last_fullsized":"2024-03-11T19:51:56.595922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531793+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:51:56.531793+0000","last_peered":"2024-03-11T19:51:56.531793+0000","last_clean":"2024-03-11T19:51:56.531793+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:51:56.531793+0000","last_undegraded":"2024-03-11T19:51:56.531793+0000","last_fullsized":"2024-03-11T19:51:56.531793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709765+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:51:56.709765+0000","last_peered":"2024-03-11T19:51:56.709765+0000","last_clean":"2024-03-11T19:51:56.709765+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:51:56.709765+0000","last_undegraded":"2024-03-11T19:51:56.709765+0000","last_fullsized":"2024-03-11T19:51:56.709765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708988+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:51:56.708988+0000","last_peered":"2024-03-11T19:51:56.708988+0000","last_clean":"2024-03-11T19:51:56.708988+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:51:56.708988+0000","last_undegraded":"2024-03-11T19:51:56.708988+0000","last_fullsized":"2024-03-11T19:51:56.708988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708926+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:51:56.708926+0000","last_peered":"2024-03-11T19:51:56.708926+0000","last_clean":"2024-03-11T19:51:56.708926+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:51:56.708926+0000","last_undegraded":"2024-03-11T19:51:56.708926+0000","last_fullsized":"2024-03-11T19:51:56.708926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-11T19:51:55.523700+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:51:55.523700+0000","last_peered":"2024-03-11T19:51:55.523700+0000","last_clean":"2024-03-11T19:51:55.523700+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:51:55.523700+0000","last_undegraded":"2024-03-11T19:51:55.523700+0000","last_fullsized":"2024-03-11T19:51:55.523700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716335+0000","last_change":"2024-03-11T19:51:56.716335+0000","last_active":"2024-03-11T19:51:56.716335+0000","last_peered":"2024-03-11T19:51:56.716335+0000","last_clean":"2024-03-11T19:51:56.716335+0000","last_became_active":"2024-03-11T19:51:56.714786+0000","last_became_peered":"2024-03-11T19:51:56.714786+0000","last_unstale":"2024-03-11T19:51:56.716335+0000","last_undegraded":"2024-03-11T19:51:56.716335+0000","last_fullsized":"2024-03-11T19:51:56.716335+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:39:36.271156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709691+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:51:56.709691+0000","last_peered":"2024-03-11T19:51:56.709691+0000","last_clean":"2024-03-11T19:51:56.709691+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:51:56.709691+0000","last_undegraded":"2024-03-11T19:51:56.709691+0000","last_fullsized":"2024-03-11T19:51:56.709691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709510+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:51:56.709510+0000","last_peered":"2024-03-11T19:51:56.709510+0000","last_clean":"2024-03-11T19:51:56.709510+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:51:56.709510+0000","last_undegraded":"2024-03-11T19:51:56.709510+0000","last_fullsized":"2024-03-11T19:51:56.709510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527102+0000","last_change":"2024-03-11T19:50:55.488762+0000","last_active":"2024-03-11T19:51:56.527102+0000","last_peered":"2024-03-11T19:51:56.527102+0000","last_clean":"2024-03-11T19:51:56.527102+0000","last_became_active":"2024-03-11T19:50:55.488324+0000","last_became_peered":"2024-03-11T19:50:55.488324+0000","last_unstale":"2024-03-11T19:51:56.527102+0000","last_undegraded":"2024-03-11T19:51:56.527102+0000","last_fullsized":"2024-03-11T19:51:56.527102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:36.522393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596090+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:51:56.596090+0000","last_peered":"2024-03-11T19:51:56.596090+0000","last_clean":"2024-03-11T19:51:56.596090+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:51:56.596090+0000","last_undegraded":"2024-03-11T19:51:56.596090+0000","last_fullsized":"2024-03-11T19:51:56.596090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525884+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:51:56.525884+0000","last_peered":"2024-03-11T19:51:56.525884+0000","last_clean":"2024-03-11T19:51:56.525884+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:51:56.525884+0000","last_undegraded":"2024-03-11T19:51:56.525884+0000","last_fullsized":"2024-03-11T19:51:56.525884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709202+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:51:56.709202+0000","last_peered":"2024-03-11T19:51:56.709202+0000","last_clean":"2024-03-11T19:51:56.709202+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:51:56.709202+0000","last_undegraded":"2024-03-11T19:51:56.709202+0000","last_fullsized":"2024-03-11T19:51:56.709202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530667+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:51:56.530667+0000","last_peered":"2024-03-11T19:51:56.530667+0000","last_clean":"2024-03-11T19:51:56.530667+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:51:56.530667+0000","last_undegraded":"2024-03-11T19:51:56.530667+0000","last_fullsized":"2024-03-11T19:51:56.530667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709904+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:51:56.709904+0000","last_peered":"2024-03-11T19:51:56.709904+0000","last_clean":"2024-03-11T19:51:56.709904+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:51:56.709904+0000","last_undegraded":"2024-03-11T19:51:56.709904+0000","last_fullsized":"2024-03-11T19:51:56.709904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":559,"num_read_kb":352,"num_write":172,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4842,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":537362,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1187840,"data_stored":1191260,"data_compressed":67061,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518662,"data_compressed":23640,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":537474,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":1134078,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":532752,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":585374,"data_compressed":28246,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":524716,"data_compressed":26632,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":595302,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532946,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":539088,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1129134,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62136,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62640,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:52:03.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:52:03.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:03 smithi107 ceph-mon[31712]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:03.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:03 smithi107 ceph-mon[31712]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 779 B/s wr, 5 op/s 2024-03-11T19:52:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:03 smithi122 ceph-mon[36397]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:03 smithi122 ceph-mon[36397]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 779 B/s wr, 5 op/s 2024-03-11T19:52:03.781 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:03 smithi145 ceph-mon[36730]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:03 smithi145 ceph-mon[36730]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 779 B/s wr, 5 op/s 2024-03-11T19:52:05.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:05 smithi107 ceph-mon[31712]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 577 B/s wr, 6 op/s 2024-03-11T19:52:05.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:05 smithi122 ceph-mon[36397]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 577 B/s wr, 6 op/s 2024-03-11T19:52:05.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:05 smithi145 ceph-mon[36730]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 577 B/s wr, 6 op/s 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.079+0000 7f197be00700 1 -- 172.21.15.107:0/563819323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974112b20 msgr2=0x7f1974114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.079+0000 7f197be00700 1 --2- 172.21.15.107:0/563819323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974112b20 0x7f1974114f10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b0a0 tx=0x7f196c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 -- 172.21.15.107:0/563819323 shutdown_connections 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 --2- 172.21.15.107:0/563819323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974112b20 0x7f1974114f10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 --2- 172.21.15.107:0/563819323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f1974076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 --2- 172.21.15.107:0/563819323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974076ac0 0x7f19741125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 -- 172.21.15.107:0/563819323 >> 172.21.15.107:0/563819323 conn(0x7f19740ff970 msgr2=0x7f1974101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 -- 172.21.15.107:0/563819323 shutdown_connections 2024-03-11T19:52:06.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.080+0000 7f197be00700 1 -- 172.21.15.107:0/563819323 wait complete. 2024-03-11T19:52:06.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.081+0000 7f197be00700 1 Processor -- start 2024-03-11T19:52:06.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.081+0000 7f197be00700 1 -- start start 2024-03-11T19:52:06.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 0x7f19741a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974112b20 0x7f197419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197419ee40 con 0x7f19740761a0 2024-03-11T19:52:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197419efb0 con 0x7f1974076ac0 2024-03-11T19:52:06.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f197be00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197419f170 con 0x7f1974112b20 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f1979400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f1979400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44732/0 (socket says 172.21.15.107:44732) 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f1979400700 1 -- 172.21.15.107:0/2969065148 learned_addr learned my addr 172.21.15.107:0/2969065148 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f1978a00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 0x7f19741a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.082+0000 7f1979e00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974112b20 0x7f197419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:06.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 -- 172.21.15.107:0/2969065148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974112b20 msgr2=0x7f197419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974112b20 0x7f197419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 -- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 msgr2=0x7f19741a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 -- 172.21.15.107:0/2969065148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c009d00 con 0x7f1974076ac0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1979400700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1978a00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 0x7f19741a4b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000c9c0 con 0x7f1974076ac0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1960010430 con 0x7f1974076ac0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197419f450 con 0x7f1974076ac0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960018680 con 0x7f1974076ac0 2024-03-11T19:52:06.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.083+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19741ab640 con 0x7f1974076ac0 2024-03-11T19:52:06.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.085+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197410dd40 con 0x7f1974076ac0 2024-03-11T19:52:06.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.086+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1960010910 con 0x7f1974076ac0 2024-03-11T19:52:06.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.088+0000 7f1972000700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 0x7f195c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:06.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.088+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1960014070 con 0x7f1974076ac0 2024-03-11T19:52:06.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.088+0000 7f1979400700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 0x7f195c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:06.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.089+0000 7f1979400700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 0x7f195c0a4000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1968011550 tx=0x7f1968009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.092+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1960052d20 con 0x7f1974076ac0 2024-03-11T19:52:06.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.309+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f197419fd30 con 0x7f195c0a1b40 2024-03-11T19:52:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.338+0000 7f1972000700 1 -- 172.21.15.107:0/2969065148 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340862 (secure 0 0 0) 0x7f197419fd30 con 0x7f195c0a1b40 2024-03-11T19:52:06.352 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:06.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 msgr2=0x7f195c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 0x7f195c0a4000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1968011550 tx=0x7f1968009260 comp rx=0 tx=0).stop 2024-03-11T19:52:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 msgr2=0x7f19741a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 0x7f19741a4b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000ed00 comp rx=0 tx=0).stop 2024-03-11T19:52:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 shutdown_connections 2024-03-11T19:52:06.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f195c0a1b40 0x7f195c0a4000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1974112b20 0x7f197419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19740761a0 0x7f19741a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 --2- 172.21.15.107:0/2969065148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1974076ac0 0x7f19741a4b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.344+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 >> 172.21.15.107:0/2969065148 conn(0x7f19740ff970 msgr2=0x7f1974101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.345+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 shutdown_connections 2024-03-11T19:52:06.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:06.345+0000 7f197be00700 1 -- 172.21.15.107:0/2969065148 wait complete. 2024-03-11T19:52:06.358 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:52:07.216 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-03-11T19:52:04.379883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":715,"num_read_kb":481,"num_write":355,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182000,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353660,"kb_used_data":12716,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588164,"statfs":{"total":1151940427776,"available":1151578279936,"internally_reserved":0,"allocated":13021184,"data_stored":8395910,"data_compressed":467862,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":27,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.861303"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710068+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:51:56.710068+0000","last_peered":"2024-03-11T19:51:56.710068+0000","last_clean":"2024-03-11T19:51:56.710068+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:51:56.710068+0000","last_undegraded":"2024-03-11T19:51:56.710068+0000","last_fullsized":"2024-03-11T19:51:56.710068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539955+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:51:56.539955+0000","last_peered":"2024-03-11T19:51:56.539955+0000","last_clean":"2024-03-11T19:51:56.539955+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:51:56.539955+0000","last_undegraded":"2024-03-11T19:51:56.539955+0000","last_fullsized":"2024-03-11T19:51:56.539955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538895+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:51:56.538895+0000","last_peered":"2024-03-11T19:51:56.538895+0000","last_clean":"2024-03-11T19:51:56.538895+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:51:56.538895+0000","last_undegraded":"2024-03-11T19:51:56.538895+0000","last_fullsized":"2024-03-11T19:51:56.538895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523239+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:51:56.523239+0000","last_peered":"2024-03-11T19:51:56.523239+0000","last_clean":"2024-03-11T19:51:56.523239+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:51:56.523239+0000","last_undegraded":"2024-03-11T19:51:56.523239+0000","last_fullsized":"2024-03-11T19:51:56.523239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522831+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:51:56.522831+0000","last_peered":"2024-03-11T19:51:56.522831+0000","last_clean":"2024-03-11T19:51:56.522831+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:51:56.522831+0000","last_undegraded":"2024-03-11T19:51:56.522831+0000","last_fullsized":"2024-03-11T19:51:56.522831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532313+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:51:56.532313+0000","last_peered":"2024-03-11T19:51:56.532313+0000","last_clean":"2024-03-11T19:51:56.532313+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:51:56.532313+0000","last_undegraded":"2024-03-11T19:51:56.532313+0000","last_fullsized":"2024-03-11T19:51:56.532313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709889+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:51:56.709889+0000","last_peered":"2024-03-11T19:51:56.709889+0000","last_clean":"2024-03-11T19:51:56.709889+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:51:56.709889+0000","last_undegraded":"2024-03-11T19:51:56.709889+0000","last_fullsized":"2024-03-11T19:51:56.709889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.524741+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:51:56.524741+0000","last_peered":"2024-03-11T19:51:56.524741+0000","last_clean":"2024-03-11T19:51:56.524741+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:51:56.524741+0000","last_undegraded":"2024-03-11T19:51:56.524741+0000","last_fullsized":"2024-03-11T19:51:56.524741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709398+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:51:56.709398+0000","last_peered":"2024-03-11T19:51:56.709398+0000","last_clean":"2024-03-11T19:51:56.709398+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:51:56.709398+0000","last_undegraded":"2024-03-11T19:51:56.709398+0000","last_fullsized":"2024-03-11T19:51:56.709398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532578+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:51:56.532578+0000","last_peered":"2024-03-11T19:51:56.532578+0000","last_clean":"2024-03-11T19:51:56.532578+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:51:56.532578+0000","last_undegraded":"2024-03-11T19:51:56.532578+0000","last_fullsized":"2024-03-11T19:51:56.532578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709875+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:51:56.709875+0000","last_peered":"2024-03-11T19:51:56.709875+0000","last_clean":"2024-03-11T19:51:56.709875+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:51:56.709875+0000","last_undegraded":"2024-03-11T19:51:56.709875+0000","last_fullsized":"2024-03-11T19:51:56.709875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523155+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:51:56.523155+0000","last_peered":"2024-03-11T19:51:56.523155+0000","last_clean":"2024-03-11T19:51:56.523155+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:51:56.523155+0000","last_undegraded":"2024-03-11T19:51:56.523155+0000","last_fullsized":"2024-03-11T19:51:56.523155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716871+0000","last_change":"2024-03-11T19:51:56.716871+0000","last_active":"2024-03-11T19:51:56.716871+0000","last_peered":"2024-03-11T19:51:56.716871+0000","last_clean":"2024-03-11T19:51:56.716871+0000","last_became_active":"2024-03-11T19:51:56.715370+0000","last_became_peered":"2024-03-11T19:51:56.715370+0000","last_unstale":"2024-03-11T19:51:56.716871+0000","last_undegraded":"2024-03-11T19:51:56.716871+0000","last_fullsized":"2024-03-11T19:51:56.716871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:13:56.734860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538332+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:51:56.538332+0000","last_peered":"2024-03-11T19:51:56.538332+0000","last_clean":"2024-03-11T19:51:56.538332+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:51:56.538332+0000","last_undegraded":"2024-03-11T19:51:56.538332+0000","last_fullsized":"2024-03-11T19:51:56.538332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538188+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:51:56.538188+0000","last_peered":"2024-03-11T19:51:56.538188+0000","last_clean":"2024-03-11T19:51:56.538188+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:51:56.538188+0000","last_undegraded":"2024-03-11T19:51:56.538188+0000","last_fullsized":"2024-03-11T19:51:56.538188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709009+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:51:56.709009+0000","last_peered":"2024-03-11T19:51:56.709009+0000","last_clean":"2024-03-11T19:51:56.709009+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:51:56.709009+0000","last_undegraded":"2024-03-11T19:51:56.709009+0000","last_fullsized":"2024-03-11T19:51:56.709009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709723+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:51:56.709723+0000","last_peered":"2024-03-11T19:51:56.709723+0000","last_clean":"2024-03-11T19:51:56.709723+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:51:56.709723+0000","last_undegraded":"2024-03-11T19:51:56.709723+0000","last_fullsized":"2024-03-11T19:51:56.709723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713076+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:51:56.713076+0000","last_peered":"2024-03-11T19:51:56.713076+0000","last_clean":"2024-03-11T19:51:56.713076+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:51:56.713076+0000","last_undegraded":"2024-03-11T19:51:56.713076+0000","last_fullsized":"2024-03-11T19:51:56.713076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709909+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:51:56.709909+0000","last_peered":"2024-03-11T19:51:56.709909+0000","last_clean":"2024-03-11T19:51:56.709909+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:51:56.709909+0000","last_undegraded":"2024-03-11T19:51:56.709909+0000","last_fullsized":"2024-03-11T19:51:56.709909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708991+0000","last_change":"2024-03-11T19:50:55.487486+0000","last_active":"2024-03-11T19:51:56.708991+0000","last_peered":"2024-03-11T19:51:56.708991+0000","last_clean":"2024-03-11T19:51:56.708991+0000","last_became_active":"2024-03-11T19:50:55.486998+0000","last_became_peered":"2024-03-11T19:50:55.486998+0000","last_unstale":"2024-03-11T19:51:56.708991+0000","last_undegraded":"2024-03-11T19:51:56.708991+0000","last_fullsized":"2024-03-11T19:51:56.708991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:39.904812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713947+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:51:56.713947+0000","last_peered":"2024-03-11T19:51:56.713947+0000","last_clean":"2024-03-11T19:51:56.713947+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:51:56.713947+0000","last_undegraded":"2024-03-11T19:51:56.713947+0000","last_fullsized":"2024-03-11T19:51:56.713947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526502+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:51:56.526502+0000","last_peered":"2024-03-11T19:51:56.526502+0000","last_clean":"2024-03-11T19:51:56.526502+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:51:56.526502+0000","last_undegraded":"2024-03-11T19:51:56.526502+0000","last_fullsized":"2024-03-11T19:51:56.526502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522659+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:51:56.522659+0000","last_peered":"2024-03-11T19:51:56.522659+0000","last_clean":"2024-03-11T19:51:56.522659+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:51:56.522659+0000","last_undegraded":"2024-03-11T19:51:56.522659+0000","last_fullsized":"2024-03-11T19:51:56.522659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527995+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:51:56.527995+0000","last_peered":"2024-03-11T19:51:56.527995+0000","last_clean":"2024-03-11T19:51:56.527995+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:51:56.527995+0000","last_undegraded":"2024-03-11T19:51:56.527995+0000","last_fullsized":"2024-03-11T19:51:56.527995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540550+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:51:56.540550+0000","last_peered":"2024-03-11T19:51:56.540550+0000","last_clean":"2024-03-11T19:51:56.540550+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:51:56.540550+0000","last_undegraded":"2024-03-11T19:51:56.540550+0000","last_fullsized":"2024-03-11T19:51:56.540550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522810+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:51:56.522810+0000","last_peered":"2024-03-11T19:51:56.522810+0000","last_clean":"2024-03-11T19:51:56.522810+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:51:56.522810+0000","last_undegraded":"2024-03-11T19:51:56.522810+0000","last_fullsized":"2024-03-11T19:51:56.522810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709813+0000","last_change":"2024-03-11T19:50:55.480971+0000","last_active":"2024-03-11T19:51:56.709813+0000","last_peered":"2024-03-11T19:51:56.709813+0000","last_clean":"2024-03-11T19:51:56.709813+0000","last_became_active":"2024-03-11T19:50:55.474612+0000","last_became_peered":"2024-03-11T19:50:55.474612+0000","last_unstale":"2024-03-11T19:51:56.709813+0000","last_undegraded":"2024-03-11T19:51:56.709813+0000","last_fullsized":"2024-03-11T19:51:56.709813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:59:47.829059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522632+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:51:56.522632+0000","last_peered":"2024-03-11T19:51:56.522632+0000","last_clean":"2024-03-11T19:51:56.522632+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:51:56.522632+0000","last_undegraded":"2024-03-11T19:51:56.522632+0000","last_fullsized":"2024-03-11T19:51:56.522632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522758+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:51:56.522758+0000","last_peered":"2024-03-11T19:51:56.522758+0000","last_clean":"2024-03-11T19:51:56.522758+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:51:56.522758+0000","last_undegraded":"2024-03-11T19:51:56.522758+0000","last_fullsized":"2024-03-11T19:51:56.522758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596489+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:51:56.596489+0000","last_peered":"2024-03-11T19:51:56.596489+0000","last_clean":"2024-03-11T19:51:56.596489+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:51:56.596489+0000","last_undegraded":"2024-03-11T19:51:56.596489+0000","last_fullsized":"2024-03-11T19:51:56.596489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709721+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:51:56.709721+0000","last_peered":"2024-03-11T19:51:56.709721+0000","last_clean":"2024-03-11T19:51:56.709721+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:51:56.709721+0000","last_undegraded":"2024-03-11T19:51:56.709721+0000","last_fullsized":"2024-03-11T19:51:56.709721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":150,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:01.603909+0000","last_change":"2024-03-11T19:50:55.509345+0000","last_active":"2024-03-11T19:52:01.603909+0000","last_peered":"2024-03-11T19:52:01.603909+0000","last_clean":"2024-03-11T19:52:01.603909+0000","last_became_active":"2024-03-11T19:50:55.505792+0000","last_became_peered":"2024-03-11T19:50:55.505792+0000","last_unstale":"2024-03-11T19:52:01.603909+0000","last_undegraded":"2024-03-11T19:52:01.603909+0000","last_fullsized":"2024-03-11T19:52:01.603909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:01:20.682572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522735+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:51:56.522735+0000","last_peered":"2024-03-11T19:51:56.522735+0000","last_clean":"2024-03-11T19:51:56.522735+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:51:56.522735+0000","last_undegraded":"2024-03-11T19:51:56.522735+0000","last_fullsized":"2024-03-11T19:51:56.522735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709487+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:51:56.709487+0000","last_peered":"2024-03-11T19:51:56.709487+0000","last_clean":"2024-03-11T19:51:56.709487+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:51:56.709487+0000","last_undegraded":"2024-03-11T19:51:56.709487+0000","last_fullsized":"2024-03-11T19:51:56.709487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532197+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:51:56.532197+0000","last_peered":"2024-03-11T19:51:56.532197+0000","last_clean":"2024-03-11T19:51:56.532197+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:51:56.532197+0000","last_undegraded":"2024-03-11T19:51:56.532197+0000","last_fullsized":"2024-03-11T19:51:56.532197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596519+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:51:56.596519+0000","last_peered":"2024-03-11T19:51:56.596519+0000","last_clean":"2024-03-11T19:51:56.596519+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:51:56.596519+0000","last_undegraded":"2024-03-11T19:51:56.596519+0000","last_fullsized":"2024-03-11T19:51:56.596519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.529353+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:51:56.529353+0000","last_peered":"2024-03-11T19:51:56.529353+0000","last_clean":"2024-03-11T19:51:56.529353+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:51:56.529353+0000","last_undegraded":"2024-03-11T19:51:56.529353+0000","last_fullsized":"2024-03-11T19:51:56.529353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711388+0000","last_change":"2024-03-11T19:51:55.547947+0000","last_active":"2024-03-11T19:51:56.711388+0000","last_peered":"2024-03-11T19:51:56.711388+0000","last_clean":"2024-03-11T19:51:56.711388+0000","last_became_active":"2024-03-11T19:51:55.547500+0000","last_became_peered":"2024-03-11T19:51:55.547500+0000","last_unstale":"2024-03-11T19:51:56.711388+0000","last_undegraded":"2024-03-11T19:51:56.711388+0000","last_fullsized":"2024-03-11T19:51:56.711388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:44:01.043269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539927+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:51:56.539927+0000","last_peered":"2024-03-11T19:51:56.539927+0000","last_clean":"2024-03-11T19:51:56.539927+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:51:56.539927+0000","last_undegraded":"2024-03-11T19:51:56.539927+0000","last_fullsized":"2024-03-11T19:51:56.539927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596062+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:51:56.596062+0000","last_peered":"2024-03-11T19:51:56.596062+0000","last_clean":"2024-03-11T19:51:56.596062+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:51:56.596062+0000","last_undegraded":"2024-03-11T19:51:56.596062+0000","last_fullsized":"2024-03-11T19:51:56.596062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712060+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:51:56.712060+0000","last_peered":"2024-03-11T19:51:56.712060+0000","last_clean":"2024-03-11T19:51:56.712060+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:51:56.712060+0000","last_undegraded":"2024-03-11T19:51:56.712060+0000","last_fullsized":"2024-03-11T19:51:56.712060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525570+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:51:56.525570+0000","last_peered":"2024-03-11T19:51:56.525570+0000","last_clean":"2024-03-11T19:51:56.525570+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:51:56.525570+0000","last_undegraded":"2024-03-11T19:51:56.525570+0000","last_fullsized":"2024-03-11T19:51:56.525570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":130,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:01.603346+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:52:01.603346+0000","last_peered":"2024-03-11T19:52:01.603346+0000","last_clean":"2024-03-11T19:52:01.603346+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:52:01.603346+0000","last_undegraded":"2024-03-11T19:52:01.603346+0000","last_fullsized":"2024-03-11T19:52:01.603346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709914+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:51:56.709914+0000","last_peered":"2024-03-11T19:51:56.709914+0000","last_clean":"2024-03-11T19:51:56.709914+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:51:56.709914+0000","last_undegraded":"2024-03-11T19:51:56.709914+0000","last_fullsized":"2024-03-11T19:51:56.709914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711767+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:51:56.711767+0000","last_peered":"2024-03-11T19:51:56.711767+0000","last_clean":"2024-03-11T19:51:56.711767+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:51:56.711767+0000","last_undegraded":"2024-03-11T19:51:56.711767+0000","last_fullsized":"2024-03-11T19:51:56.711767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709237+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:51:56.709237+0000","last_peered":"2024-03-11T19:51:56.709237+0000","last_clean":"2024-03-11T19:51:56.709237+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:51:56.709237+0000","last_undegraded":"2024-03-11T19:51:56.709237+0000","last_fullsized":"2024-03-11T19:51:56.709237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540153+0000","last_change":"2024-03-11T19:51:55.547076+0000","last_active":"2024-03-11T19:51:56.540153+0000","last_peered":"2024-03-11T19:51:56.540153+0000","last_clean":"2024-03-11T19:51:56.540153+0000","last_became_active":"2024-03-11T19:51:55.546410+0000","last_became_peered":"2024-03-11T19:51:55.546410+0000","last_unstale":"2024-03-11T19:51:56.540153+0000","last_undegraded":"2024-03-11T19:51:56.540153+0000","last_fullsized":"2024-03-11T19:51:56.540153+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:43:21.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522940+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:51:56.522940+0000","last_peered":"2024-03-11T19:51:56.522940+0000","last_clean":"2024-03-11T19:51:56.522940+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:51:56.522940+0000","last_undegraded":"2024-03-11T19:51:56.522940+0000","last_fullsized":"2024-03-11T19:51:56.522940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596354+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:51:56.596354+0000","last_peered":"2024-03-11T19:51:56.596354+0000","last_clean":"2024-03-11T19:51:56.596354+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:51:56.596354+0000","last_undegraded":"2024-03-11T19:51:56.596354+0000","last_fullsized":"2024-03-11T19:51:56.596354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532035+0000","last_change":"2024-03-11T19:50:55.475760+0000","last_active":"2024-03-11T19:51:56.532035+0000","last_peered":"2024-03-11T19:51:56.532035+0000","last_clean":"2024-03-11T19:51:56.532035+0000","last_became_active":"2024-03-11T19:50:55.475170+0000","last_became_peered":"2024-03-11T19:50:55.475170+0000","last_unstale":"2024-03-11T19:51:56.532035+0000","last_undegraded":"2024-03-11T19:51:56.532035+0000","last_fullsized":"2024-03-11T19:51:56.532035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:58.278212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":262,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:02.604539+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:52:02.604539+0000","last_peered":"2024-03-11T19:52:02.604539+0000","last_clean":"2024-03-11T19:52:02.604539+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:52:02.604539+0000","last_undegraded":"2024-03-11T19:52:02.604539+0000","last_fullsized":"2024-03-11T19:52:02.604539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":194,"num_read_kb":158,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523089+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:51:56.523089+0000","last_peered":"2024-03-11T19:51:56.523089+0000","last_clean":"2024-03-11T19:51:56.523089+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:51:56.523089+0000","last_undegraded":"2024-03-11T19:51:56.523089+0000","last_fullsized":"2024-03-11T19:51:56.523089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596828+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:51:56.596828+0000","last_peered":"2024-03-11T19:51:56.596828+0000","last_clean":"2024-03-11T19:51:56.596828+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:51:56.596828+0000","last_undegraded":"2024-03-11T19:51:56.596828+0000","last_fullsized":"2024-03-11T19:51:56.596828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709030+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:51:56.709030+0000","last_peered":"2024-03-11T19:51:56.709030+0000","last_clean":"2024-03-11T19:51:56.709030+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:51:56.709030+0000","last_undegraded":"2024-03-11T19:51:56.709030+0000","last_fullsized":"2024-03-11T19:51:56.709030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530696+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:51:56.530696+0000","last_peered":"2024-03-11T19:51:56.530696+0000","last_clean":"2024-03-11T19:51:56.530696+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:51:56.530696+0000","last_undegraded":"2024-03-11T19:51:56.530696+0000","last_fullsized":"2024-03-11T19:51:56.530696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709442+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:51:56.709442+0000","last_peered":"2024-03-11T19:51:56.709442+0000","last_clean":"2024-03-11T19:51:56.709442+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:51:56.709442+0000","last_undegraded":"2024-03-11T19:51:56.709442+0000","last_fullsized":"2024-03-11T19:51:56.709442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596343+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:51:56.596343+0000","last_peered":"2024-03-11T19:51:56.596343+0000","last_clean":"2024-03-11T19:51:56.596343+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:51:56.596343+0000","last_undegraded":"2024-03-11T19:51:56.596343+0000","last_fullsized":"2024-03-11T19:51:56.596343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710085+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:51:56.710085+0000","last_peered":"2024-03-11T19:51:56.710085+0000","last_clean":"2024-03-11T19:51:56.710085+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:51:56.710085+0000","last_undegraded":"2024-03-11T19:51:56.710085+0000","last_fullsized":"2024-03-11T19:51:56.710085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713445+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:51:56.713445+0000","last_peered":"2024-03-11T19:51:56.713445+0000","last_clean":"2024-03-11T19:51:56.713445+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:51:56.713445+0000","last_undegraded":"2024-03-11T19:51:56.713445+0000","last_fullsized":"2024-03-11T19:51:56.713445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":134,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:58.640851+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:51:58.640851+0000","last_peered":"2024-03-11T19:51:58.640851+0000","last_clean":"2024-03-11T19:51:58.640851+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:51:58.640851+0000","last_undegraded":"2024-03-11T19:51:58.640851+0000","last_fullsized":"2024-03-11T19:51:58.640851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708887+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:51:56.708887+0000","last_peered":"2024-03-11T19:51:56.708887+0000","last_clean":"2024-03-11T19:51:56.708887+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:51:56.708887+0000","last_undegraded":"2024-03-11T19:51:56.708887+0000","last_fullsized":"2024-03-11T19:51:56.708887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531921+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:51:56.531921+0000","last_peered":"2024-03-11T19:51:56.531921+0000","last_clean":"2024-03-11T19:51:56.531921+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:51:56.531921+0000","last_undegraded":"2024-03-11T19:51:56.531921+0000","last_fullsized":"2024-03-11T19:51:56.531921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522544+0000","last_change":"2024-03-11T19:50:55.807772+0000","last_active":"2024-03-11T19:51:56.522544+0000","last_peered":"2024-03-11T19:51:56.522544+0000","last_clean":"2024-03-11T19:51:56.522544+0000","last_became_active":"2024-03-11T19:50:55.794019+0000","last_became_peered":"2024-03-11T19:50:55.794019+0000","last_unstale":"2024-03-11T19:51:56.522544+0000","last_undegraded":"2024-03-11T19:51:56.522544+0000","last_fullsized":"2024-03-11T19:51:56.522544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:00.170150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596163+0000","last_change":"2024-03-11T19:50:55.793868+0000","last_active":"2024-03-11T19:51:56.596163+0000","last_peered":"2024-03-11T19:51:56.596163+0000","last_clean":"2024-03-11T19:51:56.596163+0000","last_became_active":"2024-03-11T19:50:55.793299+0000","last_became_peered":"2024-03-11T19:50:55.793299+0000","last_unstale":"2024-03-11T19:51:56.596163+0000","last_undegraded":"2024-03-11T19:51:56.596163+0000","last_fullsized":"2024-03-11T19:51:56.596163+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:40:42.039449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596033+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:51:56.596033+0000","last_peered":"2024-03-11T19:51:56.596033+0000","last_clean":"2024-03-11T19:51:56.596033+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:51:56.596033+0000","last_undegraded":"2024-03-11T19:51:56.596033+0000","last_fullsized":"2024-03-11T19:51:56.596033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523361+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:51:56.523361+0000","last_peered":"2024-03-11T19:51:56.523361+0000","last_clean":"2024-03-11T19:51:56.523361+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:51:56.523361+0000","last_undegraded":"2024-03-11T19:51:56.523361+0000","last_fullsized":"2024-03-11T19:51:56.523361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709559+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:51:56.709559+0000","last_peered":"2024-03-11T19:51:56.709559+0000","last_clean":"2024-03-11T19:51:56.709559+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:51:56.709559+0000","last_undegraded":"2024-03-11T19:51:56.709559+0000","last_fullsized":"2024-03-11T19:51:56.709559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522734+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:51:56.522734+0000","last_peered":"2024-03-11T19:51:56.522734+0000","last_clean":"2024-03-11T19:51:56.522734+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:51:56.522734+0000","last_undegraded":"2024-03-11T19:51:56.522734+0000","last_fullsized":"2024-03-11T19:51:56.522734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712674+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:51:56.712674+0000","last_peered":"2024-03-11T19:51:56.712674+0000","last_clean":"2024-03-11T19:51:56.712674+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:51:56.712674+0000","last_undegraded":"2024-03-11T19:51:56.712674+0000","last_fullsized":"2024-03-11T19:51:56.712674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532585+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:51:56.532585+0000","last_peered":"2024-03-11T19:51:56.532585+0000","last_clean":"2024-03-11T19:51:56.532585+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:51:56.532585+0000","last_undegraded":"2024-03-11T19:51:56.532585+0000","last_fullsized":"2024-03-11T19:51:56.532585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531395+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:51:56.531395+0000","last_peered":"2024-03-11T19:51:56.531395+0000","last_clean":"2024-03-11T19:51:56.531395+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:51:56.531395+0000","last_undegraded":"2024-03-11T19:51:56.531395+0000","last_fullsized":"2024-03-11T19:51:56.531395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709519+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:51:56.709519+0000","last_peered":"2024-03-11T19:51:56.709519+0000","last_clean":"2024-03-11T19:51:56.709519+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:51:56.709519+0000","last_undegraded":"2024-03-11T19:51:56.709519+0000","last_fullsized":"2024-03-11T19:51:56.709519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532690+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:51:56.532690+0000","last_peered":"2024-03-11T19:51:56.532690+0000","last_clean":"2024-03-11T19:51:56.532690+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:51:56.532690+0000","last_undegraded":"2024-03-11T19:51:56.532690+0000","last_fullsized":"2024-03-11T19:51:56.532690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532300+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:51:56.532300+0000","last_peered":"2024-03-11T19:51:56.532300+0000","last_clean":"2024-03-11T19:51:56.532300+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:51:56.532300+0000","last_undegraded":"2024-03-11T19:51:56.532300+0000","last_fullsized":"2024-03-11T19:51:56.532300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596553+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:51:56.596553+0000","last_peered":"2024-03-11T19:51:56.596553+0000","last_clean":"2024-03-11T19:51:56.596553+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:51:56.596553+0000","last_undegraded":"2024-03-11T19:51:56.596553+0000","last_fullsized":"2024-03-11T19:51:56.596553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":130,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:58.639702+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:51:58.639702+0000","last_peered":"2024-03-11T19:51:58.639702+0000","last_clean":"2024-03-11T19:51:58.639702+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:51:58.639702+0000","last_undegraded":"2024-03-11T19:51:58.639702+0000","last_fullsized":"2024-03-11T19:51:58.639702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709555+0000","last_change":"2024-03-11T19:50:55.488155+0000","last_active":"2024-03-11T19:51:56.709555+0000","last_peered":"2024-03-11T19:51:56.709555+0000","last_clean":"2024-03-11T19:51:56.709555+0000","last_became_active":"2024-03-11T19:50:55.477254+0000","last_became_peered":"2024-03-11T19:50:55.477254+0000","last_unstale":"2024-03-11T19:51:56.709555+0000","last_undegraded":"2024-03-11T19:51:56.709555+0000","last_fullsized":"2024-03-11T19:51:56.709555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:05.827200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527420+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:51:56.527420+0000","last_peered":"2024-03-11T19:51:56.527420+0000","last_clean":"2024-03-11T19:51:56.527420+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:51:56.527420+0000","last_undegraded":"2024-03-11T19:51:56.527420+0000","last_fullsized":"2024-03-11T19:51:56.527420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522985+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:51:56.522985+0000","last_peered":"2024-03-11T19:51:56.522985+0000","last_clean":"2024-03-11T19:51:56.522985+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:51:56.522985+0000","last_undegraded":"2024-03-11T19:51:56.522985+0000","last_fullsized":"2024-03-11T19:51:56.522985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708751+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:51:56.708751+0000","last_peered":"2024-03-11T19:51:56.708751+0000","last_clean":"2024-03-11T19:51:56.708751+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:51:56.708751+0000","last_undegraded":"2024-03-11T19:51:56.708751+0000","last_fullsized":"2024-03-11T19:51:56.708751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596781+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:51:56.596781+0000","last_peered":"2024-03-11T19:51:56.596781+0000","last_clean":"2024-03-11T19:51:56.596781+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:51:56.596781+0000","last_undegraded":"2024-03-11T19:51:56.596781+0000","last_fullsized":"2024-03-11T19:51:56.596781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.528022+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:51:56.528022+0000","last_peered":"2024-03-11T19:51:56.528022+0000","last_clean":"2024-03-11T19:51:56.528022+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:51:56.528022+0000","last_undegraded":"2024-03-11T19:51:56.528022+0000","last_fullsized":"2024-03-11T19:51:56.528022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708807+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:51:56.708807+0000","last_peered":"2024-03-11T19:51:56.708807+0000","last_clean":"2024-03-11T19:51:56.708807+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:51:56.708807+0000","last_undegraded":"2024-03-11T19:51:56.708807+0000","last_fullsized":"2024-03-11T19:51:56.708807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709301+0000","last_change":"2024-03-11T19:50:55.474853+0000","last_active":"2024-03-11T19:51:56.709301+0000","last_peered":"2024-03-11T19:51:56.709301+0000","last_clean":"2024-03-11T19:51:56.709301+0000","last_became_active":"2024-03-11T19:50:55.474379+0000","last_became_peered":"2024-03-11T19:50:55.474379+0000","last_unstale":"2024-03-11T19:51:56.709301+0000","last_undegraded":"2024-03-11T19:51:56.709301+0000","last_fullsized":"2024-03-11T19:51:56.709301+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:43.691094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709662+0000","last_change":"2024-03-11T19:50:55.491848+0000","last_active":"2024-03-11T19:51:56.709662+0000","last_peered":"2024-03-11T19:51:56.709662+0000","last_clean":"2024-03-11T19:51:56.709662+0000","last_became_active":"2024-03-11T19:50:55.485786+0000","last_became_peered":"2024-03-11T19:50:55.485786+0000","last_unstale":"2024-03-11T19:51:56.709662+0000","last_undegraded":"2024-03-11T19:51:56.709662+0000","last_fullsized":"2024-03-11T19:51:56.709662+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:24:41.406176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532569+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:51:56.532569+0000","last_peered":"2024-03-11T19:51:56.532569+0000","last_clean":"2024-03-11T19:51:56.532569+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:51:56.532569+0000","last_undegraded":"2024-03-11T19:51:56.532569+0000","last_fullsized":"2024-03-11T19:51:56.532569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525309+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:51:56.525309+0000","last_peered":"2024-03-11T19:51:56.525309+0000","last_clean":"2024-03-11T19:51:56.525309+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:51:56.525309+0000","last_undegraded":"2024-03-11T19:51:56.525309+0000","last_fullsized":"2024-03-11T19:51:56.525309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713087+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:51:56.713087+0000","last_peered":"2024-03-11T19:51:56.713087+0000","last_clean":"2024-03-11T19:51:56.713087+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:51:56.713087+0000","last_undegraded":"2024-03-11T19:51:56.713087+0000","last_fullsized":"2024-03-11T19:51:56.713087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526601+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:51:56.526601+0000","last_peered":"2024-03-11T19:51:56.526601+0000","last_clean":"2024-03-11T19:51:56.526601+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:51:56.526601+0000","last_undegraded":"2024-03-11T19:51:56.526601+0000","last_fullsized":"2024-03-11T19:51:56.526601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708706+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:51:56.708706+0000","last_peered":"2024-03-11T19:51:56.708706+0000","last_clean":"2024-03-11T19:51:56.708706+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:51:56.708706+0000","last_undegraded":"2024-03-11T19:51:56.708706+0000","last_fullsized":"2024-03-11T19:51:56.708706+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523340+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:51:56.523340+0000","last_peered":"2024-03-11T19:51:56.523340+0000","last_clean":"2024-03-11T19:51:56.523340+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:51:56.523340+0000","last_undegraded":"2024-03-11T19:51:56.523340+0000","last_fullsized":"2024-03-11T19:51:56.523340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540068+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:51:56.540068+0000","last_peered":"2024-03-11T19:51:56.540068+0000","last_clean":"2024-03-11T19:51:56.540068+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:51:56.540068+0000","last_undegraded":"2024-03-11T19:51:56.540068+0000","last_fullsized":"2024-03-11T19:51:56.540068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523190+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:51:56.523190+0000","last_peered":"2024-03-11T19:51:56.523190+0000","last_clean":"2024-03-11T19:51:56.523190+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:51:56.523190+0000","last_undegraded":"2024-03-11T19:51:56.523190+0000","last_fullsized":"2024-03-11T19:51:56.523190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709061+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:51:56.709061+0000","last_peered":"2024-03-11T19:51:56.709061+0000","last_clean":"2024-03-11T19:51:56.709061+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:51:56.709061+0000","last_undegraded":"2024-03-11T19:51:56.709061+0000","last_fullsized":"2024-03-11T19:51:56.709061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522822+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:51:56.522822+0000","last_peered":"2024-03-11T19:51:56.522822+0000","last_clean":"2024-03-11T19:51:56.522822+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:51:56.522822+0000","last_undegraded":"2024-03-11T19:51:56.522822+0000","last_fullsized":"2024-03-11T19:51:56.522822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713725+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:51:56.713725+0000","last_peered":"2024-03-11T19:51:56.713725+0000","last_clean":"2024-03-11T19:51:56.713725+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:51:56.713725+0000","last_undegraded":"2024-03-11T19:51:56.713725+0000","last_fullsized":"2024-03-11T19:51:56.713725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525797+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:51:56.525797+0000","last_peered":"2024-03-11T19:51:56.525797+0000","last_clean":"2024-03-11T19:51:56.525797+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:51:56.525797+0000","last_undegraded":"2024-03-11T19:51:56.525797+0000","last_fullsized":"2024-03-11T19:51:56.525797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709514+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:51:56.709514+0000","last_peered":"2024-03-11T19:51:56.709514+0000","last_clean":"2024-03-11T19:51:56.709514+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:51:56.709514+0000","last_undegraded":"2024-03-11T19:51:56.709514+0000","last_fullsized":"2024-03-11T19:51:56.709514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523006+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:51:56.523006+0000","last_peered":"2024-03-11T19:51:56.523006+0000","last_clean":"2024-03-11T19:51:56.523006+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:51:56.523006+0000","last_undegraded":"2024-03-11T19:51:56.523006+0000","last_fullsized":"2024-03-11T19:51:56.523006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708841+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:51:56.708841+0000","last_peered":"2024-03-11T19:51:56.708841+0000","last_clean":"2024-03-11T19:51:56.708841+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:51:56.708841+0000","last_undegraded":"2024-03-11T19:51:56.708841+0000","last_fullsized":"2024-03-11T19:51:56.708841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525612+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:51:56.525612+0000","last_peered":"2024-03-11T19:51:56.525612+0000","last_clean":"2024-03-11T19:51:56.525612+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:51:56.525612+0000","last_undegraded":"2024-03-11T19:51:56.525612+0000","last_fullsized":"2024-03-11T19:51:56.525612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522988+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:51:56.522988+0000","last_peered":"2024-03-11T19:51:56.522988+0000","last_clean":"2024-03-11T19:51:56.522988+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:51:56.522988+0000","last_undegraded":"2024-03-11T19:51:56.522988+0000","last_fullsized":"2024-03-11T19:51:56.522988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522717+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:51:56.522717+0000","last_peered":"2024-03-11T19:51:56.522717+0000","last_clean":"2024-03-11T19:51:56.522717+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:51:56.522717+0000","last_undegraded":"2024-03-11T19:51:56.522717+0000","last_fullsized":"2024-03-11T19:51:56.522717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709536+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:51:56.709536+0000","last_peered":"2024-03-11T19:51:56.709536+0000","last_clean":"2024-03-11T19:51:56.709536+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:51:56.709536+0000","last_undegraded":"2024-03-11T19:51:56.709536+0000","last_fullsized":"2024-03-11T19:51:56.709536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708833+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:51:56.708833+0000","last_peered":"2024-03-11T19:51:56.708833+0000","last_clean":"2024-03-11T19:51:56.708833+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:51:56.708833+0000","last_undegraded":"2024-03-11T19:51:56.708833+0000","last_fullsized":"2024-03-11T19:51:56.708833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709374+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:51:56.709374+0000","last_peered":"2024-03-11T19:51:56.709374+0000","last_clean":"2024-03-11T19:51:56.709374+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:51:56.709374+0000","last_undegraded":"2024-03-11T19:51:56.709374+0000","last_fullsized":"2024-03-11T19:51:56.709374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710015+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:51:56.710015+0000","last_peered":"2024-03-11T19:51:56.710015+0000","last_clean":"2024-03-11T19:51:56.710015+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:51:56.710015+0000","last_undegraded":"2024-03-11T19:51:56.710015+0000","last_fullsized":"2024-03-11T19:51:56.710015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538344+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:51:56.538344+0000","last_peered":"2024-03-11T19:51:56.538344+0000","last_clean":"2024-03-11T19:51:56.538344+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:51:56.538344+0000","last_undegraded":"2024-03-11T19:51:56.538344+0000","last_fullsized":"2024-03-11T19:51:56.538344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596670+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:51:56.596670+0000","last_peered":"2024-03-11T19:51:56.596670+0000","last_clean":"2024-03-11T19:51:56.596670+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:51:56.596670+0000","last_undegraded":"2024-03-11T19:51:56.596670+0000","last_fullsized":"2024-03-11T19:51:56.596670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709620+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:51:56.709620+0000","last_peered":"2024-03-11T19:51:56.709620+0000","last_clean":"2024-03-11T19:51:56.709620+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:51:56.709620+0000","last_undegraded":"2024-03-11T19:51:56.709620+0000","last_fullsized":"2024-03-11T19:51:56.709620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.595922+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:51:56.595922+0000","last_peered":"2024-03-11T19:51:56.595922+0000","last_clean":"2024-03-11T19:51:56.595922+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:51:56.595922+0000","last_undegraded":"2024-03-11T19:51:56.595922+0000","last_fullsized":"2024-03-11T19:51:56.595922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531793+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:51:56.531793+0000","last_peered":"2024-03-11T19:51:56.531793+0000","last_clean":"2024-03-11T19:51:56.531793+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:51:56.531793+0000","last_undegraded":"2024-03-11T19:51:56.531793+0000","last_fullsized":"2024-03-11T19:51:56.531793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709836+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:51:56.709836+0000","last_peered":"2024-03-11T19:51:56.709836+0000","last_clean":"2024-03-11T19:51:56.709836+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:51:56.709836+0000","last_undegraded":"2024-03-11T19:51:56.709836+0000","last_fullsized":"2024-03-11T19:51:56.709836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708988+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:51:56.708988+0000","last_peered":"2024-03-11T19:51:56.708988+0000","last_clean":"2024-03-11T19:51:56.708988+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:51:56.708988+0000","last_undegraded":"2024-03-11T19:51:56.708988+0000","last_fullsized":"2024-03-11T19:51:56.708988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708926+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:51:56.708926+0000","last_peered":"2024-03-11T19:51:56.708926+0000","last_clean":"2024-03-11T19:51:56.708926+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:51:56.708926+0000","last_undegraded":"2024-03-11T19:51:56.708926+0000","last_fullsized":"2024-03-11T19:51:56.708926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539179+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:51:56.539179+0000","last_peered":"2024-03-11T19:51:56.539179+0000","last_clean":"2024-03-11T19:51:56.539179+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:51:56.539179+0000","last_undegraded":"2024-03-11T19:51:56.539179+0000","last_fullsized":"2024-03-11T19:51:56.539179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716335+0000","last_change":"2024-03-11T19:51:56.716335+0000","last_active":"2024-03-11T19:51:56.716335+0000","last_peered":"2024-03-11T19:51:56.716335+0000","last_clean":"2024-03-11T19:51:56.716335+0000","last_became_active":"2024-03-11T19:51:56.714786+0000","last_became_peered":"2024-03-11T19:51:56.714786+0000","last_unstale":"2024-03-11T19:51:56.716335+0000","last_undegraded":"2024-03-11T19:51:56.716335+0000","last_fullsized":"2024-03-11T19:51:56.716335+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:39:36.271156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709691+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:51:56.709691+0000","last_peered":"2024-03-11T19:51:56.709691+0000","last_clean":"2024-03-11T19:51:56.709691+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:51:56.709691+0000","last_undegraded":"2024-03-11T19:51:56.709691+0000","last_fullsized":"2024-03-11T19:51:56.709691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709510+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:51:56.709510+0000","last_peered":"2024-03-11T19:51:56.709510+0000","last_clean":"2024-03-11T19:51:56.709510+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:51:56.709510+0000","last_undegraded":"2024-03-11T19:51:56.709510+0000","last_fullsized":"2024-03-11T19:51:56.709510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527102+0000","last_change":"2024-03-11T19:50:55.488762+0000","last_active":"2024-03-11T19:51:56.527102+0000","last_peered":"2024-03-11T19:51:56.527102+0000","last_clean":"2024-03-11T19:51:56.527102+0000","last_became_active":"2024-03-11T19:50:55.488324+0000","last_became_peered":"2024-03-11T19:50:55.488324+0000","last_unstale":"2024-03-11T19:51:56.527102+0000","last_undegraded":"2024-03-11T19:51:56.527102+0000","last_fullsized":"2024-03-11T19:51:56.527102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:36.522393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596090+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:51:56.596090+0000","last_peered":"2024-03-11T19:51:56.596090+0000","last_clean":"2024-03-11T19:51:56.596090+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:51:56.596090+0000","last_undegraded":"2024-03-11T19:51:56.596090+0000","last_fullsized":"2024-03-11T19:51:56.596090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525884+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:51:56.525884+0000","last_peered":"2024-03-11T19:51:56.525884+0000","last_clean":"2024-03-11T19:51:56.525884+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:51:56.525884+0000","last_undegraded":"2024-03-11T19:51:56.525884+0000","last_fullsized":"2024-03-11T19:51:56.525884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709202+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:51:56.709202+0000","last_peered":"2024-03-11T19:51:56.709202+0000","last_clean":"2024-03-11T19:51:56.709202+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:51:56.709202+0000","last_undegraded":"2024-03-11T19:51:56.709202+0000","last_fullsized":"2024-03-11T19:51:56.709202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530667+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:51:56.530667+0000","last_peered":"2024-03-11T19:51:56.530667+0000","last_clean":"2024-03-11T19:51:56.530667+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:51:56.530667+0000","last_undegraded":"2024-03-11T19:51:56.530667+0000","last_fullsized":"2024-03-11T19:51:56.530667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709904+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:51:56.709904+0000","last_peered":"2024-03-11T19:51:56.709904+0000","last_clean":"2024-03-11T19:51:56.709904+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:51:56.709904+0000","last_undegraded":"2024-03-11T19:51:56.709904+0000","last_fullsized":"2024-03-11T19:51:56.709904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":579,"num_read_kb":362,"num_write":172,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4842,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":537362,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1187840,"data_stored":1191260,"data_compressed":67061,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":537576,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":537474,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":1134078,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":532752,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1110016,"data_stored":594798,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":534140,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":595302,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532946,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":539088,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1129134,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62136,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62640,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:52:07.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph status --format=json 2024-03-11T19:52:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:07.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:07 smithi122 ceph-mon[36397]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:07.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:07 smithi122 ceph-mon[36397]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:52:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:07 smithi145 ceph-mon[36730]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:07.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:07 smithi145 ceph-mon[36730]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:52:07.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:07 smithi107 ceph-mon[31712]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:07.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:07 smithi107 ceph-mon[31712]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-11T19:52:09.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:09 smithi107 ceph-mon[31712]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 431 B/s wr, 6 op/s 2024-03-11T19:52:09.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:09 smithi122 ceph-mon[36397]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 431 B/s wr, 6 op/s 2024-03-11T19:52:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:09 smithi145 ceph-mon[36730]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 431 B/s wr, 6 op/s 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 -- 172.21.15.107:0/1871242654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 msgr2=0x7fd2fc1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 --2- 172.21.15.107:0/1871242654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc1044a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009990 tx=0x7fd2ec009ca0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 -- 172.21.15.107:0/1871242654 shutdown_connections 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 --2- 172.21.15.107:0/1871242654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc1091a0 0x7fd2fc110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 --2- 172.21.15.107:0/1871242654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc1044a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 --2- 172.21.15.107:0/1871242654 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc104a70 0x7fd2fc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 -- 172.21.15.107:0/1871242654 >> 172.21.15.107:0/1871242654 conn(0x7fd2fc0ff950 msgr2=0x7fd2fc101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 -- 172.21.15.107:0/1871242654 shutdown_connections 2024-03-11T19:52:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.139+0000 7fd304600700 1 -- 172.21.15.107:0/1871242654 wait complete. 2024-03-11T19:52:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.140+0000 7fd304600700 1 Processor -- start 2024-03-11T19:52:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.140+0000 7fd304600700 1 -- start start 2024-03-11T19:52:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc104a70 0x7fd2fc19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc1091a0 0x7fd2fc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19f2d0 con 0x7fd2fc1040c0 2024-03-11T19:52:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19f440 con 0x7fd2fc104a70 2024-03-11T19:52:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd304600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19f600 con 0x7fd2fc1091a0 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44758/0 (socket says 172.21.15.107:44758) 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301c00700 1 -- 172.21.15.107:0/4133274363 learned_addr learned my addr 172.21.15.107:0/4133274363 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd302600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc1091a0 0x7fd2fc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301200700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc104a70 0x7fd2fc19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301c00700 1 -- 172.21.15.107:0/4133274363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc1091a0 msgr2=0x7fd2fc19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:10.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.141+0000 7fd301c00700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc1091a0 0x7fd2fc19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd301c00700 1 -- 172.21.15.107:0/4133274363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc104a70 msgr2=0x7fd2fc19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd301c00700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc104a70 0x7fd2fc19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd301c00700 1 -- 172.21.15.107:0/4133274363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0095f0 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd301c00700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec005910 tx=0x7fd2ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec01d070 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2ec00b610 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0173f0 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc19f880 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.142+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1a6f70 con 0x7fd2fc1040c0 2024-03-11T19:52:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.144+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc10c460 con 0x7fd2fc1040c0 2024-03-11T19:52:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.146+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd2ec022470 con 0x7fd2fc1040c0 2024-03-11T19:52:10.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.148+0000 7fd2fa000700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 0x7fd2e00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:10.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.148+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd2ec0cea10 con 0x7fd2fc1040c0 2024-03-11T19:52:10.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.148+0000 7fd301200700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 0x7fd2e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:10.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.149+0000 7fd301200700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 0x7fd2e00a3f80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc10a700 tx=0x7fd2e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:10.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.153+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2ec05dd20 con 0x7fd2fc1040c0 2024-03-11T19:52:10.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.464+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd2fc113fc0 con 0x7fd2fc1040c0 2024-03-11T19:52:10.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.465+0000 7fd2fa000700 1 -- 172.21.15.107:0/4133274363 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1495 (secure 0 0 0) 0x7fd2ec097020 con 0x7fd2fc1040c0 2024-03-11T19:52:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:10.470 INFO:teuthology.orchestra.run.smithi107.stdout:{"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":543,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":12,"num_up_osds":12,"osd_up_since":1710186536,"num_in_osds":12,"osd_in_since":1710186517,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":657300,"bytes_used":362147840,"bytes_avail":1151578279936,"bytes_total":1151940427776,"read_bytes_sec":3070,"write_bytes_sec":85,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24517},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24529},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24533},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24547},{"filesystem_id":1,"rank":1,"name":"a","status":"up:active","gid":24583},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24590}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.107:8443/"}},"servicemap":{"epoch":28,"modified":"2024-03-11T19:51:36.366701+0000","services":{}},"progress_events":{}} 2024-03-11T19:52:10.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 msgr2=0x7fd2e00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:10.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 0x7fd2e00a3f80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc10a700 tx=0x7fd2e8009260 comp rx=0 tx=0).stop 2024-03-11T19:52:10.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 msgr2=0x7fd2fc19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:10.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec005910 tx=0x7fd2ec004730 comp rx=0 tx=0).stop 2024-03-11T19:52:10.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 shutdown_connections 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd2e00a1ac0 0x7fd2e00a3f80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2fc1091a0 0x7fd2fc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc1040c0 0x7fd2fc19a3f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 --2- 172.21.15.107:0/4133274363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2fc104a70 0x7fd2fc19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.468+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 >> 172.21.15.107:0/4133274363 conn(0x7fd2fc0ff950 msgr2=0x7fd2fc101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.469+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 shutdown_connections 2024-03-11T19:52:10.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:10.469+0000 7fd304600700 1 -- 172.21.15.107:0/4133274363 wait complete. 2024-03-11T19:52:11.342 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-03-11T19:52:11.342 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:52:11.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:11 smithi107 ceph-mon[31712]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-11T19:52:11.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/4133274363' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-11T19:52:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:11 smithi122 ceph-mon[36397]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-11T19:52:11.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:11 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/4133274363' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-11T19:52:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:11 smithi145 ceph-mon[36730]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-11T19:52:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:11 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/4133274363' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-11T19:52:11.801 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:13.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:13 smithi107 ceph-mon[31712]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:13.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:13 smithi122 ceph-mon[36397]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:13.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:13 smithi145 ceph-mon[36730]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 -- 172.21.15.107:0/848663730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 msgr2=0x7f5774108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 --2- 172.21.15.107:0/848663730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f5774108920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5768009990 tx=0x7f5768009ca0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 -- 172.21.15.107:0/848663730 shutdown_connections 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 --2- 172.21.15.107:0/848663730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 0x7f57741151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 --2- 172.21.15.107:0/848663730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f5774108920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 --2- 172.21.15.107:0/848663730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f577410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 -- 172.21.15.107:0/848663730 >> 172.21.15.107:0/848663730 conn(0x7f57740762c0 msgr2=0x7f57740786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 -- 172.21.15.107:0/848663730 shutdown_connections 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.130+0000 7f577b400700 1 -- 172.21.15.107:0/848663730 wait complete. 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.131+0000 7f577b400700 1 Processor -- start 2024-03-11T19:52:14.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.131+0000 7f577b400700 1 -- start start 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f57741a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 0x7f577419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577419eed0 con 0x7f5774108540 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577419f040 con 0x7f5774108ef0 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f577b400700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577419f200 con 0x7f577410d620 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5778a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f57741a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5773e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5773e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:59552/0 (socket says 172.21.15.107:59552) 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5773e00700 1 -- 172.21.15.107:0/829537209 learned_addr learned my addr 172.21.15.107:0/829537209 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5778a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f57741a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44778/0 (socket says 172.21.15.107:44778) 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.132+0000 7f5779400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 0x7f577419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 -- 172.21.15.107:0/829537209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 msgr2=0x7f577419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 0x7f577419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 -- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 msgr2=0x7f57741a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f57741a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 -- 172.21.15.107:0/829537209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57680095f0 con 0x7f5774108ef0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5773e00700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f576000d980 tx=0x7f576000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576000f890 con 0x7f5774108ef0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5760010460 con 0x7f5774108ef0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577419f4e0 con 0x7f5774108ef0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576000e610 con 0x7f5774108ef0 2024-03-11T19:52:14.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.133+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57741ab580 con 0x7f5774108ef0 2024-03-11T19:52:14.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.134+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57741108e0 con 0x7f5774108ef0 2024-03-11T19:52:14.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.136+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5760010940 con 0x7f5774108ef0 2024-03-11T19:52:14.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.137+0000 7f5771600700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 0x7f575c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.138+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f57600cd260 con 0x7f5774108ef0 2024-03-11T19:52:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.138+0000 7f5778a00700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 0x7f575c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.138+0000 7f5778a00700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 0x7f575c0a3f80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f576800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.142+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5760096060 con 0x7f5774108ef0 2024-03-11T19:52:14.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.353+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f577419fc70 con 0x7f575c0a1ac0 2024-03-11T19:52:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.389+0000 7f5771600700 1 -- 172.21.15.107:0/829537209 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340816 (secure 0 0 0) 0x7f577419fc70 con 0x7f575c0a1ac0 2024-03-11T19:52:14.396 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 msgr2=0x7f575c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 0x7f575c0a3f80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f576800b260 comp rx=0 tx=0).stop 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 msgr2=0x7f57741a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f576000d980 tx=0x7f576000dc90 comp rx=0 tx=0).stop 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 shutdown_connections 2024-03-11T19:52:14.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f575c0a1ac0 0x7f575c0a3f80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f577410d620 0x7f577419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5774108540 0x7f57741a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 --2- 172.21.15.107:0/829537209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5774108ef0 0x7f57741a4b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 >> 172.21.15.107:0/829537209 conn(0x7f57740762c0 msgr2=0x7f5774077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.395+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 shutdown_connections 2024-03-11T19:52:14.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:14.396+0000 7f577b400700 1 -- 172.21.15.107:0/829537209 wait complete. 2024-03-11T19:52:14.404 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:52:15.251 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-03-11T19:52:12.383855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":755,"num_read_kb":501,"num_write":355,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182000,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353660,"kb_used_data":12716,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588164,"statfs":{"total":1151940427776,"available":1151578279936,"internally_reserved":0,"allocated":13021184,"data_stored":8395910,"data_compressed":467862,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006656"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710068+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:51:56.710068+0000","last_peered":"2024-03-11T19:51:56.710068+0000","last_clean":"2024-03-11T19:51:56.710068+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:51:56.710068+0000","last_undegraded":"2024-03-11T19:51:56.710068+0000","last_fullsized":"2024-03-11T19:51:56.710068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539955+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:51:56.539955+0000","last_peered":"2024-03-11T19:51:56.539955+0000","last_clean":"2024-03-11T19:51:56.539955+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:51:56.539955+0000","last_undegraded":"2024-03-11T19:51:56.539955+0000","last_fullsized":"2024-03-11T19:51:56.539955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538895+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:51:56.538895+0000","last_peered":"2024-03-11T19:51:56.538895+0000","last_clean":"2024-03-11T19:51:56.538895+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:51:56.538895+0000","last_undegraded":"2024-03-11T19:51:56.538895+0000","last_fullsized":"2024-03-11T19:51:56.538895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523239+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:51:56.523239+0000","last_peered":"2024-03-11T19:51:56.523239+0000","last_clean":"2024-03-11T19:51:56.523239+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:51:56.523239+0000","last_undegraded":"2024-03-11T19:51:56.523239+0000","last_fullsized":"2024-03-11T19:51:56.523239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522831+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:51:56.522831+0000","last_peered":"2024-03-11T19:51:56.522831+0000","last_clean":"2024-03-11T19:51:56.522831+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:51:56.522831+0000","last_undegraded":"2024-03-11T19:51:56.522831+0000","last_fullsized":"2024-03-11T19:51:56.522831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532313+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:51:56.532313+0000","last_peered":"2024-03-11T19:51:56.532313+0000","last_clean":"2024-03-11T19:51:56.532313+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:51:56.532313+0000","last_undegraded":"2024-03-11T19:51:56.532313+0000","last_fullsized":"2024-03-11T19:51:56.532313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709889+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:51:56.709889+0000","last_peered":"2024-03-11T19:51:56.709889+0000","last_clean":"2024-03-11T19:51:56.709889+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:51:56.709889+0000","last_undegraded":"2024-03-11T19:51:56.709889+0000","last_fullsized":"2024-03-11T19:51:56.709889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.524741+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:51:56.524741+0000","last_peered":"2024-03-11T19:51:56.524741+0000","last_clean":"2024-03-11T19:51:56.524741+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:51:56.524741+0000","last_undegraded":"2024-03-11T19:51:56.524741+0000","last_fullsized":"2024-03-11T19:51:56.524741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709398+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:51:56.709398+0000","last_peered":"2024-03-11T19:51:56.709398+0000","last_clean":"2024-03-11T19:51:56.709398+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:51:56.709398+0000","last_undegraded":"2024-03-11T19:51:56.709398+0000","last_fullsized":"2024-03-11T19:51:56.709398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532578+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:51:56.532578+0000","last_peered":"2024-03-11T19:51:56.532578+0000","last_clean":"2024-03-11T19:51:56.532578+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:51:56.532578+0000","last_undegraded":"2024-03-11T19:51:56.532578+0000","last_fullsized":"2024-03-11T19:51:56.532578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709875+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:51:56.709875+0000","last_peered":"2024-03-11T19:51:56.709875+0000","last_clean":"2024-03-11T19:51:56.709875+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:51:56.709875+0000","last_undegraded":"2024-03-11T19:51:56.709875+0000","last_fullsized":"2024-03-11T19:51:56.709875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523155+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:51:56.523155+0000","last_peered":"2024-03-11T19:51:56.523155+0000","last_clean":"2024-03-11T19:51:56.523155+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:51:56.523155+0000","last_undegraded":"2024-03-11T19:51:56.523155+0000","last_fullsized":"2024-03-11T19:51:56.523155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716871+0000","last_change":"2024-03-11T19:51:56.716871+0000","last_active":"2024-03-11T19:51:56.716871+0000","last_peered":"2024-03-11T19:51:56.716871+0000","last_clean":"2024-03-11T19:51:56.716871+0000","last_became_active":"2024-03-11T19:51:56.715370+0000","last_became_peered":"2024-03-11T19:51:56.715370+0000","last_unstale":"2024-03-11T19:51:56.716871+0000","last_undegraded":"2024-03-11T19:51:56.716871+0000","last_fullsized":"2024-03-11T19:51:56.716871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:13:56.734860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538332+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:51:56.538332+0000","last_peered":"2024-03-11T19:51:56.538332+0000","last_clean":"2024-03-11T19:51:56.538332+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:51:56.538332+0000","last_undegraded":"2024-03-11T19:51:56.538332+0000","last_fullsized":"2024-03-11T19:51:56.538332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538188+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:51:56.538188+0000","last_peered":"2024-03-11T19:51:56.538188+0000","last_clean":"2024-03-11T19:51:56.538188+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:51:56.538188+0000","last_undegraded":"2024-03-11T19:51:56.538188+0000","last_fullsized":"2024-03-11T19:51:56.538188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709009+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:51:56.709009+0000","last_peered":"2024-03-11T19:51:56.709009+0000","last_clean":"2024-03-11T19:51:56.709009+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:51:56.709009+0000","last_undegraded":"2024-03-11T19:51:56.709009+0000","last_fullsized":"2024-03-11T19:51:56.709009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709723+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:51:56.709723+0000","last_peered":"2024-03-11T19:51:56.709723+0000","last_clean":"2024-03-11T19:51:56.709723+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:51:56.709723+0000","last_undegraded":"2024-03-11T19:51:56.709723+0000","last_fullsized":"2024-03-11T19:51:56.709723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713076+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:51:56.713076+0000","last_peered":"2024-03-11T19:51:56.713076+0000","last_clean":"2024-03-11T19:51:56.713076+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:51:56.713076+0000","last_undegraded":"2024-03-11T19:51:56.713076+0000","last_fullsized":"2024-03-11T19:51:56.713076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709960+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:51:56.709960+0000","last_peered":"2024-03-11T19:51:56.709960+0000","last_clean":"2024-03-11T19:51:56.709960+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:51:56.709960+0000","last_undegraded":"2024-03-11T19:51:56.709960+0000","last_fullsized":"2024-03-11T19:51:56.709960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708991+0000","last_change":"2024-03-11T19:50:55.487486+0000","last_active":"2024-03-11T19:51:56.708991+0000","last_peered":"2024-03-11T19:51:56.708991+0000","last_clean":"2024-03-11T19:51:56.708991+0000","last_became_active":"2024-03-11T19:50:55.486998+0000","last_became_peered":"2024-03-11T19:50:55.486998+0000","last_unstale":"2024-03-11T19:51:56.708991+0000","last_undegraded":"2024-03-11T19:51:56.708991+0000","last_fullsized":"2024-03-11T19:51:56.708991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:39.904812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713947+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:51:56.713947+0000","last_peered":"2024-03-11T19:51:56.713947+0000","last_clean":"2024-03-11T19:51:56.713947+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:51:56.713947+0000","last_undegraded":"2024-03-11T19:51:56.713947+0000","last_fullsized":"2024-03-11T19:51:56.713947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526502+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:51:56.526502+0000","last_peered":"2024-03-11T19:51:56.526502+0000","last_clean":"2024-03-11T19:51:56.526502+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:51:56.526502+0000","last_undegraded":"2024-03-11T19:51:56.526502+0000","last_fullsized":"2024-03-11T19:51:56.526502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522659+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:51:56.522659+0000","last_peered":"2024-03-11T19:51:56.522659+0000","last_clean":"2024-03-11T19:51:56.522659+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:51:56.522659+0000","last_undegraded":"2024-03-11T19:51:56.522659+0000","last_fullsized":"2024-03-11T19:51:56.522659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527995+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:51:56.527995+0000","last_peered":"2024-03-11T19:51:56.527995+0000","last_clean":"2024-03-11T19:51:56.527995+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:51:56.527995+0000","last_undegraded":"2024-03-11T19:51:56.527995+0000","last_fullsized":"2024-03-11T19:51:56.527995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540550+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:51:56.540550+0000","last_peered":"2024-03-11T19:51:56.540550+0000","last_clean":"2024-03-11T19:51:56.540550+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:51:56.540550+0000","last_undegraded":"2024-03-11T19:51:56.540550+0000","last_fullsized":"2024-03-11T19:51:56.540550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522810+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:51:56.522810+0000","last_peered":"2024-03-11T19:51:56.522810+0000","last_clean":"2024-03-11T19:51:56.522810+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:51:56.522810+0000","last_undegraded":"2024-03-11T19:51:56.522810+0000","last_fullsized":"2024-03-11T19:51:56.522810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709813+0000","last_change":"2024-03-11T19:50:55.480971+0000","last_active":"2024-03-11T19:51:56.709813+0000","last_peered":"2024-03-11T19:51:56.709813+0000","last_clean":"2024-03-11T19:51:56.709813+0000","last_became_active":"2024-03-11T19:50:55.474612+0000","last_became_peered":"2024-03-11T19:50:55.474612+0000","last_unstale":"2024-03-11T19:51:56.709813+0000","last_undegraded":"2024-03-11T19:51:56.709813+0000","last_fullsized":"2024-03-11T19:51:56.709813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:59:47.829059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522632+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:51:56.522632+0000","last_peered":"2024-03-11T19:51:56.522632+0000","last_clean":"2024-03-11T19:51:56.522632+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:51:56.522632+0000","last_undegraded":"2024-03-11T19:51:56.522632+0000","last_fullsized":"2024-03-11T19:51:56.522632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522758+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:51:56.522758+0000","last_peered":"2024-03-11T19:51:56.522758+0000","last_clean":"2024-03-11T19:51:56.522758+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:51:56.522758+0000","last_undegraded":"2024-03-11T19:51:56.522758+0000","last_fullsized":"2024-03-11T19:51:56.522758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596489+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:51:56.596489+0000","last_peered":"2024-03-11T19:51:56.596489+0000","last_clean":"2024-03-11T19:51:56.596489+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:51:56.596489+0000","last_undegraded":"2024-03-11T19:51:56.596489+0000","last_fullsized":"2024-03-11T19:51:56.596489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709721+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:51:56.709721+0000","last_peered":"2024-03-11T19:51:56.709721+0000","last_clean":"2024-03-11T19:51:56.709721+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:51:56.709721+0000","last_undegraded":"2024-03-11T19:51:56.709721+0000","last_fullsized":"2024-03-11T19:51:56.709721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:06.611018+0000","last_change":"2024-03-11T19:50:55.509345+0000","last_active":"2024-03-11T19:52:06.611018+0000","last_peered":"2024-03-11T19:52:06.611018+0000","last_clean":"2024-03-11T19:52:06.611018+0000","last_became_active":"2024-03-11T19:50:55.505792+0000","last_became_peered":"2024-03-11T19:50:55.505792+0000","last_unstale":"2024-03-11T19:52:06.611018+0000","last_undegraded":"2024-03-11T19:52:06.611018+0000","last_fullsized":"2024-03-11T19:52:06.611018+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:01:20.682572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522735+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:51:56.522735+0000","last_peered":"2024-03-11T19:51:56.522735+0000","last_clean":"2024-03-11T19:51:56.522735+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:51:56.522735+0000","last_undegraded":"2024-03-11T19:51:56.522735+0000","last_fullsized":"2024-03-11T19:51:56.522735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709487+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:51:56.709487+0000","last_peered":"2024-03-11T19:51:56.709487+0000","last_clean":"2024-03-11T19:51:56.709487+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:51:56.709487+0000","last_undegraded":"2024-03-11T19:51:56.709487+0000","last_fullsized":"2024-03-11T19:51:56.709487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532197+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:51:56.532197+0000","last_peered":"2024-03-11T19:51:56.532197+0000","last_clean":"2024-03-11T19:51:56.532197+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:51:56.532197+0000","last_undegraded":"2024-03-11T19:51:56.532197+0000","last_fullsized":"2024-03-11T19:51:56.532197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596519+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:51:56.596519+0000","last_peered":"2024-03-11T19:51:56.596519+0000","last_clean":"2024-03-11T19:51:56.596519+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:51:56.596519+0000","last_undegraded":"2024-03-11T19:51:56.596519+0000","last_fullsized":"2024-03-11T19:51:56.596519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709685+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:51:56.709685+0000","last_peered":"2024-03-11T19:51:56.709685+0000","last_clean":"2024-03-11T19:51:56.709685+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:51:56.709685+0000","last_undegraded":"2024-03-11T19:51:56.709685+0000","last_fullsized":"2024-03-11T19:51:56.709685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.529353+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:51:56.529353+0000","last_peered":"2024-03-11T19:51:56.529353+0000","last_clean":"2024-03-11T19:51:56.529353+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:51:56.529353+0000","last_undegraded":"2024-03-11T19:51:56.529353+0000","last_fullsized":"2024-03-11T19:51:56.529353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711388+0000","last_change":"2024-03-11T19:51:55.547947+0000","last_active":"2024-03-11T19:51:56.711388+0000","last_peered":"2024-03-11T19:51:56.711388+0000","last_clean":"2024-03-11T19:51:56.711388+0000","last_became_active":"2024-03-11T19:51:55.547500+0000","last_became_peered":"2024-03-11T19:51:55.547500+0000","last_unstale":"2024-03-11T19:51:56.711388+0000","last_undegraded":"2024-03-11T19:51:56.711388+0000","last_fullsized":"2024-03-11T19:51:56.711388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:44:01.043269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539927+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:51:56.539927+0000","last_peered":"2024-03-11T19:51:56.539927+0000","last_clean":"2024-03-11T19:51:56.539927+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:51:56.539927+0000","last_undegraded":"2024-03-11T19:51:56.539927+0000","last_fullsized":"2024-03-11T19:51:56.539927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596062+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:51:56.596062+0000","last_peered":"2024-03-11T19:51:56.596062+0000","last_clean":"2024-03-11T19:51:56.596062+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:51:56.596062+0000","last_undegraded":"2024-03-11T19:51:56.596062+0000","last_fullsized":"2024-03-11T19:51:56.596062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712060+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:51:56.712060+0000","last_peered":"2024-03-11T19:51:56.712060+0000","last_clean":"2024-03-11T19:51:56.712060+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:51:56.712060+0000","last_undegraded":"2024-03-11T19:51:56.712060+0000","last_fullsized":"2024-03-11T19:51:56.712060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525570+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:51:56.525570+0000","last_peered":"2024-03-11T19:51:56.525570+0000","last_clean":"2024-03-11T19:51:56.525570+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:51:56.525570+0000","last_undegraded":"2024-03-11T19:51:56.525570+0000","last_fullsized":"2024-03-11T19:51:56.525570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":135,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:06.610432+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:52:06.610432+0000","last_peered":"2024-03-11T19:52:06.610432+0000","last_clean":"2024-03-11T19:52:06.610432+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:52:06.610432+0000","last_undegraded":"2024-03-11T19:52:06.610432+0000","last_fullsized":"2024-03-11T19:52:06.610432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709914+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:51:56.709914+0000","last_peered":"2024-03-11T19:51:56.709914+0000","last_clean":"2024-03-11T19:51:56.709914+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:51:56.709914+0000","last_undegraded":"2024-03-11T19:51:56.709914+0000","last_fullsized":"2024-03-11T19:51:56.709914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711767+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:51:56.711767+0000","last_peered":"2024-03-11T19:51:56.711767+0000","last_clean":"2024-03-11T19:51:56.711767+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:51:56.711767+0000","last_undegraded":"2024-03-11T19:51:56.711767+0000","last_fullsized":"2024-03-11T19:51:56.711767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709237+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:51:56.709237+0000","last_peered":"2024-03-11T19:51:56.709237+0000","last_clean":"2024-03-11T19:51:56.709237+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:51:56.709237+0000","last_undegraded":"2024-03-11T19:51:56.709237+0000","last_fullsized":"2024-03-11T19:51:56.709237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540153+0000","last_change":"2024-03-11T19:51:55.547076+0000","last_active":"2024-03-11T19:51:56.540153+0000","last_peered":"2024-03-11T19:51:56.540153+0000","last_clean":"2024-03-11T19:51:56.540153+0000","last_became_active":"2024-03-11T19:51:55.546410+0000","last_became_peered":"2024-03-11T19:51:55.546410+0000","last_unstale":"2024-03-11T19:51:56.540153+0000","last_undegraded":"2024-03-11T19:51:56.540153+0000","last_fullsized":"2024-03-11T19:51:56.540153+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:43:21.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522940+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:51:56.522940+0000","last_peered":"2024-03-11T19:51:56.522940+0000","last_clean":"2024-03-11T19:51:56.522940+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:51:56.522940+0000","last_undegraded":"2024-03-11T19:51:56.522940+0000","last_fullsized":"2024-03-11T19:51:56.522940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596354+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:51:56.596354+0000","last_peered":"2024-03-11T19:51:56.596354+0000","last_clean":"2024-03-11T19:51:56.596354+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:51:56.596354+0000","last_undegraded":"2024-03-11T19:51:56.596354+0000","last_fullsized":"2024-03-11T19:51:56.596354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709680+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:51:56.709680+0000","last_peered":"2024-03-11T19:51:56.709680+0000","last_clean":"2024-03-11T19:51:56.709680+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:51:56.709680+0000","last_undegraded":"2024-03-11T19:51:56.709680+0000","last_fullsized":"2024-03-11T19:51:56.709680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532035+0000","last_change":"2024-03-11T19:50:55.475760+0000","last_active":"2024-03-11T19:51:56.532035+0000","last_peered":"2024-03-11T19:51:56.532035+0000","last_clean":"2024-03-11T19:51:56.532035+0000","last_became_active":"2024-03-11T19:50:55.475170+0000","last_became_peered":"2024-03-11T19:50:55.475170+0000","last_unstale":"2024-03-11T19:51:56.532035+0000","last_undegraded":"2024-03-11T19:51:56.532035+0000","last_fullsized":"2024-03-11T19:51:56.532035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:58.278212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":272,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:07.611823+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:52:07.611823+0000","last_peered":"2024-03-11T19:52:07.611823+0000","last_clean":"2024-03-11T19:52:07.611823+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:52:07.611823+0000","last_undegraded":"2024-03-11T19:52:07.611823+0000","last_fullsized":"2024-03-11T19:52:07.611823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":163,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523089+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:51:56.523089+0000","last_peered":"2024-03-11T19:51:56.523089+0000","last_clean":"2024-03-11T19:51:56.523089+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:51:56.523089+0000","last_undegraded":"2024-03-11T19:51:56.523089+0000","last_fullsized":"2024-03-11T19:51:56.523089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596828+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:51:56.596828+0000","last_peered":"2024-03-11T19:51:56.596828+0000","last_clean":"2024-03-11T19:51:56.596828+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:51:56.596828+0000","last_undegraded":"2024-03-11T19:51:56.596828+0000","last_fullsized":"2024-03-11T19:51:56.596828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709030+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:51:56.709030+0000","last_peered":"2024-03-11T19:51:56.709030+0000","last_clean":"2024-03-11T19:51:56.709030+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:51:56.709030+0000","last_undegraded":"2024-03-11T19:51:56.709030+0000","last_fullsized":"2024-03-11T19:51:56.709030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530696+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:51:56.530696+0000","last_peered":"2024-03-11T19:51:56.530696+0000","last_clean":"2024-03-11T19:51:56.530696+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:51:56.530696+0000","last_undegraded":"2024-03-11T19:51:56.530696+0000","last_fullsized":"2024-03-11T19:51:56.530696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709442+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:51:56.709442+0000","last_peered":"2024-03-11T19:51:56.709442+0000","last_clean":"2024-03-11T19:51:56.709442+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:51:56.709442+0000","last_undegraded":"2024-03-11T19:51:56.709442+0000","last_fullsized":"2024-03-11T19:51:56.709442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596343+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:51:56.596343+0000","last_peered":"2024-03-11T19:51:56.596343+0000","last_clean":"2024-03-11T19:51:56.596343+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:51:56.596343+0000","last_undegraded":"2024-03-11T19:51:56.596343+0000","last_fullsized":"2024-03-11T19:51:56.596343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710085+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:51:56.710085+0000","last_peered":"2024-03-11T19:51:56.710085+0000","last_clean":"2024-03-11T19:51:56.710085+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:51:56.710085+0000","last_undegraded":"2024-03-11T19:51:56.710085+0000","last_fullsized":"2024-03-11T19:51:56.710085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713445+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:51:56.713445+0000","last_peered":"2024-03-11T19:51:56.713445+0000","last_clean":"2024-03-11T19:51:56.713445+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:51:56.713445+0000","last_undegraded":"2024-03-11T19:51:56.713445+0000","last_fullsized":"2024-03-11T19:51:56.713445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":144,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:08.664455+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:52:08.664455+0000","last_peered":"2024-03-11T19:52:08.664455+0000","last_clean":"2024-03-11T19:52:08.664455+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:52:08.664455+0000","last_undegraded":"2024-03-11T19:52:08.664455+0000","last_fullsized":"2024-03-11T19:52:08.664455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708887+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:51:56.708887+0000","last_peered":"2024-03-11T19:51:56.708887+0000","last_clean":"2024-03-11T19:51:56.708887+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:51:56.708887+0000","last_undegraded":"2024-03-11T19:51:56.708887+0000","last_fullsized":"2024-03-11T19:51:56.708887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531921+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:51:56.531921+0000","last_peered":"2024-03-11T19:51:56.531921+0000","last_clean":"2024-03-11T19:51:56.531921+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:51:56.531921+0000","last_undegraded":"2024-03-11T19:51:56.531921+0000","last_fullsized":"2024-03-11T19:51:56.531921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":75,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522544+0000","last_change":"2024-03-11T19:50:55.807772+0000","last_active":"2024-03-11T19:51:56.522544+0000","last_peered":"2024-03-11T19:51:56.522544+0000","last_clean":"2024-03-11T19:51:56.522544+0000","last_became_active":"2024-03-11T19:50:55.794019+0000","last_became_peered":"2024-03-11T19:50:55.794019+0000","last_unstale":"2024-03-11T19:51:56.522544+0000","last_undegraded":"2024-03-11T19:51:56.522544+0000","last_fullsized":"2024-03-11T19:51:56.522544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:00.170150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596163+0000","last_change":"2024-03-11T19:50:55.793868+0000","last_active":"2024-03-11T19:51:56.596163+0000","last_peered":"2024-03-11T19:51:56.596163+0000","last_clean":"2024-03-11T19:51:56.596163+0000","last_became_active":"2024-03-11T19:50:55.793299+0000","last_became_peered":"2024-03-11T19:50:55.793299+0000","last_unstale":"2024-03-11T19:51:56.596163+0000","last_undegraded":"2024-03-11T19:51:56.596163+0000","last_fullsized":"2024-03-11T19:51:56.596163+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:40:42.039449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596033+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:51:56.596033+0000","last_peered":"2024-03-11T19:51:56.596033+0000","last_clean":"2024-03-11T19:51:56.596033+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:51:56.596033+0000","last_undegraded":"2024-03-11T19:51:56.596033+0000","last_fullsized":"2024-03-11T19:51:56.596033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523392+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:51:56.523392+0000","last_peered":"2024-03-11T19:51:56.523392+0000","last_clean":"2024-03-11T19:51:56.523392+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:51:56.523392+0000","last_undegraded":"2024-03-11T19:51:56.523392+0000","last_fullsized":"2024-03-11T19:51:56.523392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709559+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:51:56.709559+0000","last_peered":"2024-03-11T19:51:56.709559+0000","last_clean":"2024-03-11T19:51:56.709559+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:51:56.709559+0000","last_undegraded":"2024-03-11T19:51:56.709559+0000","last_fullsized":"2024-03-11T19:51:56.709559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522834+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:51:56.522834+0000","last_peered":"2024-03-11T19:51:56.522834+0000","last_clean":"2024-03-11T19:51:56.522834+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:51:56.522834+0000","last_undegraded":"2024-03-11T19:51:56.522834+0000","last_fullsized":"2024-03-11T19:51:56.522834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712674+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:51:56.712674+0000","last_peered":"2024-03-11T19:51:56.712674+0000","last_clean":"2024-03-11T19:51:56.712674+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:51:56.712674+0000","last_undegraded":"2024-03-11T19:51:56.712674+0000","last_fullsized":"2024-03-11T19:51:56.712674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532585+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:51:56.532585+0000","last_peered":"2024-03-11T19:51:56.532585+0000","last_clean":"2024-03-11T19:51:56.532585+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:51:56.532585+0000","last_undegraded":"2024-03-11T19:51:56.532585+0000","last_fullsized":"2024-03-11T19:51:56.532585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531529+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:51:56.531529+0000","last_peered":"2024-03-11T19:51:56.531529+0000","last_clean":"2024-03-11T19:51:56.531529+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:51:56.531529+0000","last_undegraded":"2024-03-11T19:51:56.531529+0000","last_fullsized":"2024-03-11T19:51:56.531529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709519+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:51:56.709519+0000","last_peered":"2024-03-11T19:51:56.709519+0000","last_clean":"2024-03-11T19:51:56.709519+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:51:56.709519+0000","last_undegraded":"2024-03-11T19:51:56.709519+0000","last_fullsized":"2024-03-11T19:51:56.709519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532690+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:51:56.532690+0000","last_peered":"2024-03-11T19:51:56.532690+0000","last_clean":"2024-03-11T19:51:56.532690+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:51:56.532690+0000","last_undegraded":"2024-03-11T19:51:56.532690+0000","last_fullsized":"2024-03-11T19:51:56.532690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532450+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:51:56.532450+0000","last_peered":"2024-03-11T19:51:56.532450+0000","last_clean":"2024-03-11T19:51:56.532450+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:51:56.532450+0000","last_undegraded":"2024-03-11T19:51:56.532450+0000","last_fullsized":"2024-03-11T19:51:56.532450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596553+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:51:56.596553+0000","last_peered":"2024-03-11T19:51:56.596553+0000","last_clean":"2024-03-11T19:51:56.596553+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:51:56.596553+0000","last_undegraded":"2024-03-11T19:51:56.596553+0000","last_fullsized":"2024-03-11T19:51:56.596553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":140,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:07.661162+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:52:07.661162+0000","last_peered":"2024-03-11T19:52:07.661162+0000","last_clean":"2024-03-11T19:52:07.661162+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:52:07.661162+0000","last_undegraded":"2024-03-11T19:52:07.661162+0000","last_fullsized":"2024-03-11T19:52:07.661162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709620+0000","last_change":"2024-03-11T19:50:55.488155+0000","last_active":"2024-03-11T19:51:56.709620+0000","last_peered":"2024-03-11T19:51:56.709620+0000","last_clean":"2024-03-11T19:51:56.709620+0000","last_became_active":"2024-03-11T19:50:55.477254+0000","last_became_peered":"2024-03-11T19:50:55.477254+0000","last_unstale":"2024-03-11T19:51:56.709620+0000","last_undegraded":"2024-03-11T19:51:56.709620+0000","last_fullsized":"2024-03-11T19:51:56.709620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:05.827200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527420+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:51:56.527420+0000","last_peered":"2024-03-11T19:51:56.527420+0000","last_clean":"2024-03-11T19:51:56.527420+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:51:56.527420+0000","last_undegraded":"2024-03-11T19:51:56.527420+0000","last_fullsized":"2024-03-11T19:51:56.527420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522985+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:51:56.522985+0000","last_peered":"2024-03-11T19:51:56.522985+0000","last_clean":"2024-03-11T19:51:56.522985+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:51:56.522985+0000","last_undegraded":"2024-03-11T19:51:56.522985+0000","last_fullsized":"2024-03-11T19:51:56.522985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708751+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:51:56.708751+0000","last_peered":"2024-03-11T19:51:56.708751+0000","last_clean":"2024-03-11T19:51:56.708751+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:51:56.708751+0000","last_undegraded":"2024-03-11T19:51:56.708751+0000","last_fullsized":"2024-03-11T19:51:56.708751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596781+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:51:56.596781+0000","last_peered":"2024-03-11T19:51:56.596781+0000","last_clean":"2024-03-11T19:51:56.596781+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:51:56.596781+0000","last_undegraded":"2024-03-11T19:51:56.596781+0000","last_fullsized":"2024-03-11T19:51:56.596781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.528427+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:51:56.528427+0000","last_peered":"2024-03-11T19:51:56.528427+0000","last_clean":"2024-03-11T19:51:56.528427+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:51:56.528427+0000","last_undegraded":"2024-03-11T19:51:56.528427+0000","last_fullsized":"2024-03-11T19:51:56.528427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708807+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:51:56.708807+0000","last_peered":"2024-03-11T19:51:56.708807+0000","last_clean":"2024-03-11T19:51:56.708807+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:51:56.708807+0000","last_undegraded":"2024-03-11T19:51:56.708807+0000","last_fullsized":"2024-03-11T19:51:56.708807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709363+0000","last_change":"2024-03-11T19:50:55.474853+0000","last_active":"2024-03-11T19:51:56.709363+0000","last_peered":"2024-03-11T19:51:56.709363+0000","last_clean":"2024-03-11T19:51:56.709363+0000","last_became_active":"2024-03-11T19:50:55.474379+0000","last_became_peered":"2024-03-11T19:50:55.474379+0000","last_unstale":"2024-03-11T19:51:56.709363+0000","last_undegraded":"2024-03-11T19:51:56.709363+0000","last_fullsized":"2024-03-11T19:51:56.709363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:43.691094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709662+0000","last_change":"2024-03-11T19:50:55.491848+0000","last_active":"2024-03-11T19:51:56.709662+0000","last_peered":"2024-03-11T19:51:56.709662+0000","last_clean":"2024-03-11T19:51:56.709662+0000","last_became_active":"2024-03-11T19:50:55.485786+0000","last_became_peered":"2024-03-11T19:50:55.485786+0000","last_unstale":"2024-03-11T19:51:56.709662+0000","last_undegraded":"2024-03-11T19:51:56.709662+0000","last_fullsized":"2024-03-11T19:51:56.709662+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:24:41.406176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532724+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:51:56.532724+0000","last_peered":"2024-03-11T19:51:56.532724+0000","last_clean":"2024-03-11T19:51:56.532724+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:51:56.532724+0000","last_undegraded":"2024-03-11T19:51:56.532724+0000","last_fullsized":"2024-03-11T19:51:56.532724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525309+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:51:56.525309+0000","last_peered":"2024-03-11T19:51:56.525309+0000","last_clean":"2024-03-11T19:51:56.525309+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:51:56.525309+0000","last_undegraded":"2024-03-11T19:51:56.525309+0000","last_fullsized":"2024-03-11T19:51:56.525309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713268+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:51:56.713268+0000","last_peered":"2024-03-11T19:51:56.713268+0000","last_clean":"2024-03-11T19:51:56.713268+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:51:56.713268+0000","last_undegraded":"2024-03-11T19:51:56.713268+0000","last_fullsized":"2024-03-11T19:51:56.713268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526748+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:51:56.526748+0000","last_peered":"2024-03-11T19:51:56.526748+0000","last_clean":"2024-03-11T19:51:56.526748+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:51:56.526748+0000","last_undegraded":"2024-03-11T19:51:56.526748+0000","last_fullsized":"2024-03-11T19:51:56.526748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708772+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:51:56.708772+0000","last_peered":"2024-03-11T19:51:56.708772+0000","last_clean":"2024-03-11T19:51:56.708772+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:51:56.708772+0000","last_undegraded":"2024-03-11T19:51:56.708772+0000","last_fullsized":"2024-03-11T19:51:56.708772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523380+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:51:56.523380+0000","last_peered":"2024-03-11T19:51:56.523380+0000","last_clean":"2024-03-11T19:51:56.523380+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:51:56.523380+0000","last_undegraded":"2024-03-11T19:51:56.523380+0000","last_fullsized":"2024-03-11T19:51:56.523380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540214+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:51:56.540214+0000","last_peered":"2024-03-11T19:51:56.540214+0000","last_clean":"2024-03-11T19:51:56.540214+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:51:56.540214+0000","last_undegraded":"2024-03-11T19:51:56.540214+0000","last_fullsized":"2024-03-11T19:51:56.540214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523190+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:51:56.523190+0000","last_peered":"2024-03-11T19:51:56.523190+0000","last_clean":"2024-03-11T19:51:56.523190+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:51:56.523190+0000","last_undegraded":"2024-03-11T19:51:56.523190+0000","last_fullsized":"2024-03-11T19:51:56.523190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709129+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:51:56.709129+0000","last_peered":"2024-03-11T19:51:56.709129+0000","last_clean":"2024-03-11T19:51:56.709129+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:51:56.709129+0000","last_undegraded":"2024-03-11T19:51:56.709129+0000","last_fullsized":"2024-03-11T19:51:56.709129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522893+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:51:56.522893+0000","last_peered":"2024-03-11T19:51:56.522893+0000","last_clean":"2024-03-11T19:51:56.522893+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:51:56.522893+0000","last_undegraded":"2024-03-11T19:51:56.522893+0000","last_fullsized":"2024-03-11T19:51:56.522893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713725+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:51:56.713725+0000","last_peered":"2024-03-11T19:51:56.713725+0000","last_clean":"2024-03-11T19:51:56.713725+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:51:56.713725+0000","last_undegraded":"2024-03-11T19:51:56.713725+0000","last_fullsized":"2024-03-11T19:51:56.713725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525952+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:51:56.525952+0000","last_peered":"2024-03-11T19:51:56.525952+0000","last_clean":"2024-03-11T19:51:56.525952+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:51:56.525952+0000","last_undegraded":"2024-03-11T19:51:56.525952+0000","last_fullsized":"2024-03-11T19:51:56.525952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709572+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:51:56.709572+0000","last_peered":"2024-03-11T19:51:56.709572+0000","last_clean":"2024-03-11T19:51:56.709572+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:51:56.709572+0000","last_undegraded":"2024-03-11T19:51:56.709572+0000","last_fullsized":"2024-03-11T19:51:56.709572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523006+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:51:56.523006+0000","last_peered":"2024-03-11T19:51:56.523006+0000","last_clean":"2024-03-11T19:51:56.523006+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:51:56.523006+0000","last_undegraded":"2024-03-11T19:51:56.523006+0000","last_fullsized":"2024-03-11T19:51:56.523006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708920+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:51:56.708920+0000","last_peered":"2024-03-11T19:51:56.708920+0000","last_clean":"2024-03-11T19:51:56.708920+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:51:56.708920+0000","last_undegraded":"2024-03-11T19:51:56.708920+0000","last_fullsized":"2024-03-11T19:51:56.708920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525706+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:51:56.525706+0000","last_peered":"2024-03-11T19:51:56.525706+0000","last_clean":"2024-03-11T19:51:56.525706+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:51:56.525706+0000","last_undegraded":"2024-03-11T19:51:56.525706+0000","last_fullsized":"2024-03-11T19:51:56.525706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522988+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:51:56.522988+0000","last_peered":"2024-03-11T19:51:56.522988+0000","last_clean":"2024-03-11T19:51:56.522988+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:51:56.522988+0000","last_undegraded":"2024-03-11T19:51:56.522988+0000","last_fullsized":"2024-03-11T19:51:56.522988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522773+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:51:56.522773+0000","last_peered":"2024-03-11T19:51:56.522773+0000","last_clean":"2024-03-11T19:51:56.522773+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:51:56.522773+0000","last_undegraded":"2024-03-11T19:51:56.522773+0000","last_fullsized":"2024-03-11T19:51:56.522773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709598+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:51:56.709598+0000","last_peered":"2024-03-11T19:51:56.709598+0000","last_clean":"2024-03-11T19:51:56.709598+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:51:56.709598+0000","last_undegraded":"2024-03-11T19:51:56.709598+0000","last_fullsized":"2024-03-11T19:51:56.709598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708895+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:51:56.708895+0000","last_peered":"2024-03-11T19:51:56.708895+0000","last_clean":"2024-03-11T19:51:56.708895+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:51:56.708895+0000","last_undegraded":"2024-03-11T19:51:56.708895+0000","last_fullsized":"2024-03-11T19:51:56.708895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709432+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:51:56.709432+0000","last_peered":"2024-03-11T19:51:56.709432+0000","last_clean":"2024-03-11T19:51:56.709432+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:51:56.709432+0000","last_undegraded":"2024-03-11T19:51:56.709432+0000","last_fullsized":"2024-03-11T19:51:56.709432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710064+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:51:56.710064+0000","last_peered":"2024-03-11T19:51:56.710064+0000","last_clean":"2024-03-11T19:51:56.710064+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:51:56.710064+0000","last_undegraded":"2024-03-11T19:51:56.710064+0000","last_fullsized":"2024-03-11T19:51:56.710064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538344+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:51:56.538344+0000","last_peered":"2024-03-11T19:51:56.538344+0000","last_clean":"2024-03-11T19:51:56.538344+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:51:56.538344+0000","last_undegraded":"2024-03-11T19:51:56.538344+0000","last_fullsized":"2024-03-11T19:51:56.538344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596670+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:51:56.596670+0000","last_peered":"2024-03-11T19:51:56.596670+0000","last_clean":"2024-03-11T19:51:56.596670+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:51:56.596670+0000","last_undegraded":"2024-03-11T19:51:56.596670+0000","last_fullsized":"2024-03-11T19:51:56.596670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709707+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:51:56.709707+0000","last_peered":"2024-03-11T19:51:56.709707+0000","last_clean":"2024-03-11T19:51:56.709707+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:51:56.709707+0000","last_undegraded":"2024-03-11T19:51:56.709707+0000","last_fullsized":"2024-03-11T19:51:56.709707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.595922+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:51:56.595922+0000","last_peered":"2024-03-11T19:51:56.595922+0000","last_clean":"2024-03-11T19:51:56.595922+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:51:56.595922+0000","last_undegraded":"2024-03-11T19:51:56.595922+0000","last_fullsized":"2024-03-11T19:51:56.595922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531929+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:51:56.531929+0000","last_peered":"2024-03-11T19:51:56.531929+0000","last_clean":"2024-03-11T19:51:56.531929+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:51:56.531929+0000","last_undegraded":"2024-03-11T19:51:56.531929+0000","last_fullsized":"2024-03-11T19:51:56.531929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709750+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:51:56.709750+0000","last_peered":"2024-03-11T19:51:56.709750+0000","last_clean":"2024-03-11T19:51:56.709750+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:51:56.709750+0000","last_undegraded":"2024-03-11T19:51:56.709750+0000","last_fullsized":"2024-03-11T19:51:56.709750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709836+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:51:56.709836+0000","last_peered":"2024-03-11T19:51:56.709836+0000","last_clean":"2024-03-11T19:51:56.709836+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:51:56.709836+0000","last_undegraded":"2024-03-11T19:51:56.709836+0000","last_fullsized":"2024-03-11T19:51:56.709836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708988+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:51:56.708988+0000","last_peered":"2024-03-11T19:51:56.708988+0000","last_clean":"2024-03-11T19:51:56.708988+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:51:56.708988+0000","last_undegraded":"2024-03-11T19:51:56.708988+0000","last_fullsized":"2024-03-11T19:51:56.708988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708926+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:51:56.708926+0000","last_peered":"2024-03-11T19:51:56.708926+0000","last_clean":"2024-03-11T19:51:56.708926+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:51:56.708926+0000","last_undegraded":"2024-03-11T19:51:56.708926+0000","last_fullsized":"2024-03-11T19:51:56.708926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539179+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:51:56.539179+0000","last_peered":"2024-03-11T19:51:56.539179+0000","last_clean":"2024-03-11T19:51:56.539179+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:51:56.539179+0000","last_undegraded":"2024-03-11T19:51:56.539179+0000","last_fullsized":"2024-03-11T19:51:56.539179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.718416+0000","last_change":"2024-03-11T19:51:56.716335+0000","last_active":"2024-03-11T19:51:56.718416+0000","last_peered":"2024-03-11T19:51:56.718416+0000","last_clean":"2024-03-11T19:51:56.718416+0000","last_became_active":"2024-03-11T19:51:56.714786+0000","last_became_peered":"2024-03-11T19:51:56.714786+0000","last_unstale":"2024-03-11T19:51:56.718416+0000","last_undegraded":"2024-03-11T19:51:56.718416+0000","last_fullsized":"2024-03-11T19:51:56.718416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:39:36.271156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709769+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:51:56.709769+0000","last_peered":"2024-03-11T19:51:56.709769+0000","last_clean":"2024-03-11T19:51:56.709769+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:51:56.709769+0000","last_undegraded":"2024-03-11T19:51:56.709769+0000","last_fullsized":"2024-03-11T19:51:56.709769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709567+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:51:56.709567+0000","last_peered":"2024-03-11T19:51:56.709567+0000","last_clean":"2024-03-11T19:51:56.709567+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:51:56.709567+0000","last_undegraded":"2024-03-11T19:51:56.709567+0000","last_fullsized":"2024-03-11T19:51:56.709567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527266+0000","last_change":"2024-03-11T19:50:55.488762+0000","last_active":"2024-03-11T19:51:56.527266+0000","last_peered":"2024-03-11T19:51:56.527266+0000","last_clean":"2024-03-11T19:51:56.527266+0000","last_became_active":"2024-03-11T19:50:55.488324+0000","last_became_peered":"2024-03-11T19:50:55.488324+0000","last_unstale":"2024-03-11T19:51:56.527266+0000","last_undegraded":"2024-03-11T19:51:56.527266+0000","last_fullsized":"2024-03-11T19:51:56.527266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:36.522393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596155+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:51:56.596155+0000","last_peered":"2024-03-11T19:51:56.596155+0000","last_clean":"2024-03-11T19:51:56.596155+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:51:56.596155+0000","last_undegraded":"2024-03-11T19:51:56.596155+0000","last_fullsized":"2024-03-11T19:51:56.596155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526030+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:51:56.526030+0000","last_peered":"2024-03-11T19:51:56.526030+0000","last_clean":"2024-03-11T19:51:56.526030+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:51:56.526030+0000","last_undegraded":"2024-03-11T19:51:56.526030+0000","last_fullsized":"2024-03-11T19:51:56.526030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709253+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:51:56.709253+0000","last_peered":"2024-03-11T19:51:56.709253+0000","last_clean":"2024-03-11T19:51:56.709253+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:51:56.709253+0000","last_undegraded":"2024-03-11T19:51:56.709253+0000","last_fullsized":"2024-03-11T19:51:56.709253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530667+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:51:56.530667+0000","last_peered":"2024-03-11T19:51:56.530667+0000","last_clean":"2024-03-11T19:51:56.530667+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:51:56.530667+0000","last_undegraded":"2024-03-11T19:51:56.530667+0000","last_fullsized":"2024-03-11T19:51:56.530667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709904+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:51:56.709904+0000","last_peered":"2024-03-11T19:51:56.709904+0000","last_clean":"2024-03-11T19:51:56.709904+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:51:56.709904+0000","last_undegraded":"2024-03-11T19:51:56.709904+0000","last_fullsized":"2024-03-11T19:51:56.709904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":619,"num_read_kb":382,"num_write":172,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4842,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":537362,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1187840,"data_stored":1191260,"data_compressed":67061,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":537576,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":537474,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":1134078,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":532752,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1110016,"data_stored":594798,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":534140,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":595302,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532946,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":539088,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1129134,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62136,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62640,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:52:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:15 smithi122 ceph-mon[36397]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:15.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:15 smithi122 ceph-mon[36397]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:15.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:15 smithi145 ceph-mon[36730]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:15.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:15 smithi145 ceph-mon[36730]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:15 smithi107 ceph-mon[31712]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:15.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:15 smithi107 ceph-mon[31712]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:17.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:17 smithi122 ceph-mon[36397]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:17.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:17 smithi145 ceph-mon[36730]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:17.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:17 smithi107 ceph-mon[31712]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:18.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:52:18.699 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:19.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:19 smithi107 ceph-mon[31712]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:19.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:19 smithi122 ceph-mon[36397]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:19.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:19 smithi145 ceph-mon[36730]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 -- 172.21.15.107:0/2521530567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc101cc0 msgr2=0x7f43dc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 --2- 172.21.15.107:0/2521530567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc101cc0 0x7f43dc102140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43c80099c0 tx=0x7f43c8009cd0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 -- 172.21.15.107:0/2521530567 shutdown_connections 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 --2- 172.21.15.107:0/2521530567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc102680 0x7f43dc10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 --2- 172.21.15.107:0/2521530567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc107ca0 0x7f43dc108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 --2- 172.21.15.107:0/2521530567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc101cc0 0x7f43dc102140 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.974+0000 7f43e2c00700 1 -- 172.21.15.107:0/2521530567 >> 172.21.15.107:0/2521530567 conn(0x7f43dc078da0 msgr2=0x7f43dc0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.975+0000 7f43e2c00700 1 -- 172.21.15.107:0/2521530567 shutdown_connections 2024-03-11T19:52:20.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.975+0000 7f43e2c00700 1 -- 172.21.15.107:0/2521530567 wait complete. 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 Processor -- start 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 -- start start 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc102680 0x7f43dc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc107ca0 0x7f43dc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f2b0 con 0x7f43dc102680 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f420 con 0x7f43dc101cc0 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.976+0000 7f43e2c00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc19f5e0 con 0x7f43dc107ca0 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:37234/0 (socket says 172.21.15.107:37234) 2024-03-11T19:52:20.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 -- 172.21.15.107:0/4132156488 learned_addr learned my addr 172.21.15.107:0/4132156488 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43db400700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc102680 0x7f43dc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43e0c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc107ca0 0x7f43dc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 -- 172.21.15.107:0/4132156488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc107ca0 msgr2=0x7f43dc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc107ca0 0x7f43dc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 -- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc102680 msgr2=0x7f43dc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc102680 0x7f43dc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:20.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.977+0000 7f43dbe00700 1 -- 172.21.15.107:0/4132156488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80095f0 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43dbe00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00e9a0 tx=0x7f43cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc009790 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43cc010430 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc19f8c0 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc018680 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.978+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc1a7120 con 0x7f43dc101cc0 2024-03-11T19:52:20.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.979+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc068930 con 0x7f43dc101cc0 2024-03-11T19:52:20.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.981+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f43cc010910 con 0x7f43dc101cc0 2024-03-11T19:52:20.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.982+0000 7f43d8c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 0x7f43c00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:20.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.983+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f43cc014070 con 0x7f43dc101cc0 2024-03-11T19:52:20.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.983+0000 7f43db400700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 0x7f43c00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:20.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.983+0000 7f43db400700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 0x7f43c00a3f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43c8000c00 tx=0x7f43c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:20.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:20.987+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43cc052d20 con 0x7f43dc101cc0 2024-03-11T19:52:21.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.204+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43dc111d50 con 0x7f43c00a1ac0 2024-03-11T19:52:21.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.219+0000 7f43d8c00700 1 -- 172.21.15.107:0/4132156488 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340819 (secure 0 0 0) 0x7f43dc111d50 con 0x7f43c00a1ac0 2024-03-11T19:52:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 msgr2=0x7f43c00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 0x7f43c00a3f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43c8000c00 tx=0x7f43c8005fb0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 msgr2=0x7f43dc19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00e9a0 tx=0x7f43cc00ecb0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 shutdown_connections 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f43c00a1ac0 0x7f43c00a3f80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43dc107ca0 0x7f43dc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.226+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43dc101cc0 0x7f43dc19a3d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.227+0000 7f43e2c00700 1 --2- 172.21.15.107:0/4132156488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc102680 0x7f43dc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.227+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 >> 172.21.15.107:0/4132156488 conn(0x7f43dc078da0 msgr2=0x7f43dc10d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.227+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 shutdown_connections 2024-03-11T19:52:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:21.227+0000 7f43e2c00700 1 -- 172.21.15.107:0/4132156488 wait complete. 2024-03-11T19:52:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-11T19:52:21.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:21 smithi122 ceph-mon[36397]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:21.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:21 smithi145 ceph-mon[36730]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:21.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:21 smithi107 ceph-mon[31712]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:22.122 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-03-11T19:52:20.390704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657300,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":815,"num_read_kb":531,"num_write":355,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182000,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353660,"kb_used_data":12716,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588164,"statfs":{"total":1151940427776,"available":1151578279936,"internally_reserved":0,"allocated":13021184,"data_stored":8395910,"data_compressed":467862,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008599"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710068+0000","last_change":"2024-03-11T19:49:55.222067+0000","last_active":"2024-03-11T19:51:56.710068+0000","last_peered":"2024-03-11T19:51:56.710068+0000","last_clean":"2024-03-11T19:51:56.710068+0000","last_became_active":"2024-03-11T19:49:55.221710+0000","last_became_peered":"2024-03-11T19:49:55.221710+0000","last_unstale":"2024-03-11T19:51:56.710068+0000","last_undegraded":"2024-03-11T19:51:56.710068+0000","last_fullsized":"2024-03-11T19:51:56.710068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:25.397399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539955+0000","last_change":"2024-03-11T19:49:55.219794+0000","last_active":"2024-03-11T19:51:56.539955+0000","last_peered":"2024-03-11T19:51:56.539955+0000","last_clean":"2024-03-11T19:51:56.539955+0000","last_became_active":"2024-03-11T19:49:55.219468+0000","last_became_peered":"2024-03-11T19:49:55.219468+0000","last_unstale":"2024-03-11T19:51:56.539955+0000","last_undegraded":"2024-03-11T19:51:56.539955+0000","last_fullsized":"2024-03-11T19:51:56.539955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:15.968771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538895+0000","last_change":"2024-03-11T19:49:55.220860+0000","last_active":"2024-03-11T19:51:56.538895+0000","last_peered":"2024-03-11T19:51:56.538895+0000","last_clean":"2024-03-11T19:51:56.538895+0000","last_became_active":"2024-03-11T19:49:55.220478+0000","last_became_peered":"2024-03-11T19:49:55.220478+0000","last_unstale":"2024-03-11T19:51:56.538895+0000","last_undegraded":"2024-03-11T19:51:56.538895+0000","last_fullsized":"2024-03-11T19:51:56.538895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:06:43.684780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523239+0000","last_change":"2024-03-11T19:49:55.216509+0000","last_active":"2024-03-11T19:51:56.523239+0000","last_peered":"2024-03-11T19:51:56.523239+0000","last_clean":"2024-03-11T19:51:56.523239+0000","last_became_active":"2024-03-11T19:49:55.216216+0000","last_became_peered":"2024-03-11T19:49:55.216216+0000","last_unstale":"2024-03-11T19:51:56.523239+0000","last_undegraded":"2024-03-11T19:51:56.523239+0000","last_fullsized":"2024-03-11T19:51:56.523239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:17:31.876871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522831+0000","last_change":"2024-03-11T19:49:55.234939+0000","last_active":"2024-03-11T19:51:56.522831+0000","last_peered":"2024-03-11T19:51:56.522831+0000","last_clean":"2024-03-11T19:51:56.522831+0000","last_became_active":"2024-03-11T19:49:55.234587+0000","last_became_peered":"2024-03-11T19:49:55.234587+0000","last_unstale":"2024-03-11T19:51:56.522831+0000","last_undegraded":"2024-03-11T19:51:56.522831+0000","last_fullsized":"2024-03-11T19:51:56.522831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:38:37.966748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532313+0000","last_change":"2024-03-11T19:49:55.229532+0000","last_active":"2024-03-11T19:51:56.532313+0000","last_peered":"2024-03-11T19:51:56.532313+0000","last_clean":"2024-03-11T19:51:56.532313+0000","last_became_active":"2024-03-11T19:49:55.229198+0000","last_became_peered":"2024-03-11T19:49:55.229198+0000","last_unstale":"2024-03-11T19:51:56.532313+0000","last_undegraded":"2024-03-11T19:51:56.532313+0000","last_fullsized":"2024-03-11T19:51:56.532313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:16:02.857163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709889+0000","last_change":"2024-03-11T19:49:55.228431+0000","last_active":"2024-03-11T19:51:56.709889+0000","last_peered":"2024-03-11T19:51:56.709889+0000","last_clean":"2024-03-11T19:51:56.709889+0000","last_became_active":"2024-03-11T19:49:55.228156+0000","last_became_peered":"2024-03-11T19:49:55.228156+0000","last_unstale":"2024-03-11T19:51:56.709889+0000","last_undegraded":"2024-03-11T19:51:56.709889+0000","last_fullsized":"2024-03-11T19:51:56.709889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:11:54.529311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.524741+0000","last_change":"2024-03-11T19:49:55.237127+0000","last_active":"2024-03-11T19:51:56.524741+0000","last_peered":"2024-03-11T19:51:56.524741+0000","last_clean":"2024-03-11T19:51:56.524741+0000","last_became_active":"2024-03-11T19:49:55.236216+0000","last_became_peered":"2024-03-11T19:49:55.236216+0000","last_unstale":"2024-03-11T19:51:56.524741+0000","last_undegraded":"2024-03-11T19:51:56.524741+0000","last_fullsized":"2024-03-11T19:51:56.524741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:11:32.047237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709398+0000","last_change":"2024-03-11T19:49:55.209571+0000","last_active":"2024-03-11T19:51:56.709398+0000","last_peered":"2024-03-11T19:51:56.709398+0000","last_clean":"2024-03-11T19:51:56.709398+0000","last_became_active":"2024-03-11T19:49:55.209254+0000","last_became_peered":"2024-03-11T19:49:55.209254+0000","last_unstale":"2024-03-11T19:51:56.709398+0000","last_undegraded":"2024-03-11T19:51:56.709398+0000","last_fullsized":"2024-03-11T19:51:56.709398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:37:14.828694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532578+0000","last_change":"2024-03-11T19:49:55.212873+0000","last_active":"2024-03-11T19:51:56.532578+0000","last_peered":"2024-03-11T19:51:56.532578+0000","last_clean":"2024-03-11T19:51:56.532578+0000","last_became_active":"2024-03-11T19:49:55.212548+0000","last_became_peered":"2024-03-11T19:49:55.212548+0000","last_unstale":"2024-03-11T19:51:56.532578+0000","last_undegraded":"2024-03-11T19:51:56.532578+0000","last_fullsized":"2024-03-11T19:51:56.532578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:40.489451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709935+0000","last_change":"2024-03-11T19:49:55.212128+0000","last_active":"2024-03-11T19:51:56.709935+0000","last_peered":"2024-03-11T19:51:56.709935+0000","last_clean":"2024-03-11T19:51:56.709935+0000","last_became_active":"2024-03-11T19:49:55.211756+0000","last_became_peered":"2024-03-11T19:49:55.211756+0000","last_unstale":"2024-03-11T19:51:56.709935+0000","last_undegraded":"2024-03-11T19:51:56.709935+0000","last_fullsized":"2024-03-11T19:51:56.709935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:25.190350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523219+0000","last_change":"2024-03-11T19:49:55.216561+0000","last_active":"2024-03-11T19:51:56.523219+0000","last_peered":"2024-03-11T19:51:56.523219+0000","last_clean":"2024-03-11T19:51:56.523219+0000","last_became_active":"2024-03-11T19:49:55.215606+0000","last_became_peered":"2024-03-11T19:49:55.215606+0000","last_unstale":"2024-03-11T19:51:56.523219+0000","last_undegraded":"2024-03-11T19:51:56.523219+0000","last_fullsized":"2024-03-11T19:51:56.523219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:51:37.381619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.716871+0000","last_change":"2024-03-11T19:51:56.716871+0000","last_active":"2024-03-11T19:51:56.716871+0000","last_peered":"2024-03-11T19:51:56.716871+0000","last_clean":"2024-03-11T19:51:56.716871+0000","last_became_active":"2024-03-11T19:51:56.715370+0000","last_became_peered":"2024-03-11T19:51:56.715370+0000","last_unstale":"2024-03-11T19:51:56.716871+0000","last_undegraded":"2024-03-11T19:51:56.716871+0000","last_fullsized":"2024-03-11T19:51:56.716871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:13:56.734860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538470+0000","last_change":"2024-03-11T19:49:55.220029+0000","last_active":"2024-03-11T19:51:56.538470+0000","last_peered":"2024-03-11T19:51:56.538470+0000","last_clean":"2024-03-11T19:51:56.538470+0000","last_became_active":"2024-03-11T19:49:55.219709+0000","last_became_peered":"2024-03-11T19:49:55.219709+0000","last_unstale":"2024-03-11T19:51:56.538470+0000","last_undegraded":"2024-03-11T19:51:56.538470+0000","last_fullsized":"2024-03-11T19:51:56.538470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:39:55.093025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538396+0000","last_change":"2024-03-11T19:49:55.222112+0000","last_active":"2024-03-11T19:51:56.538396+0000","last_peered":"2024-03-11T19:51:56.538396+0000","last_clean":"2024-03-11T19:51:56.538396+0000","last_became_active":"2024-03-11T19:49:55.221447+0000","last_became_peered":"2024-03-11T19:49:55.221447+0000","last_unstale":"2024-03-11T19:51:56.538396+0000","last_undegraded":"2024-03-11T19:51:56.538396+0000","last_fullsized":"2024-03-11T19:51:56.538396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:19:43.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709092+0000","last_change":"2024-03-11T19:49:55.213078+0000","last_active":"2024-03-11T19:51:56.709092+0000","last_peered":"2024-03-11T19:51:56.709092+0000","last_clean":"2024-03-11T19:51:56.709092+0000","last_became_active":"2024-03-11T19:49:55.212779+0000","last_became_peered":"2024-03-11T19:49:55.212779+0000","last_unstale":"2024-03-11T19:51:56.709092+0000","last_undegraded":"2024-03-11T19:51:56.709092+0000","last_fullsized":"2024-03-11T19:51:56.709092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:38.550566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709776+0000","last_change":"2024-03-11T19:49:55.222143+0000","last_active":"2024-03-11T19:51:56.709776+0000","last_peered":"2024-03-11T19:51:56.709776+0000","last_clean":"2024-03-11T19:51:56.709776+0000","last_became_active":"2024-03-11T19:49:55.221737+0000","last_became_peered":"2024-03-11T19:49:55.221737+0000","last_unstale":"2024-03-11T19:51:56.709776+0000","last_undegraded":"2024-03-11T19:51:56.709776+0000","last_fullsized":"2024-03-11T19:51:56.709776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:36.804333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713280+0000","last_change":"2024-03-11T19:49:55.229785+0000","last_active":"2024-03-11T19:51:56.713280+0000","last_peered":"2024-03-11T19:51:56.713280+0000","last_clean":"2024-03-11T19:51:56.713280+0000","last_became_active":"2024-03-11T19:49:55.229389+0000","last_became_peered":"2024-03-11T19:49:55.229389+0000","last_unstale":"2024-03-11T19:51:56.713280+0000","last_undegraded":"2024-03-11T19:51:56.713280+0000","last_fullsized":"2024-03-11T19:51:56.713280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:40.535281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709960+0000","last_change":"2024-03-11T19:49:55.238360+0000","last_active":"2024-03-11T19:51:56.709960+0000","last_peered":"2024-03-11T19:51:56.709960+0000","last_clean":"2024-03-11T19:51:56.709960+0000","last_became_active":"2024-03-11T19:49:55.238071+0000","last_became_peered":"2024-03-11T19:49:55.238071+0000","last_unstale":"2024-03-11T19:51:56.709960+0000","last_undegraded":"2024-03-11T19:51:56.709960+0000","last_fullsized":"2024-03-11T19:51:56.709960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:28.365487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708991+0000","last_change":"2024-03-11T19:50:55.487486+0000","last_active":"2024-03-11T19:51:56.708991+0000","last_peered":"2024-03-11T19:51:56.708991+0000","last_clean":"2024-03-11T19:51:56.708991+0000","last_became_active":"2024-03-11T19:50:55.486998+0000","last_became_peered":"2024-03-11T19:50:55.486998+0000","last_unstale":"2024-03-11T19:51:56.708991+0000","last_undegraded":"2024-03-11T19:51:56.708991+0000","last_fullsized":"2024-03-11T19:51:56.708991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:39.904812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713947+0000","last_change":"2024-03-11T19:49:54.717820+0000","last_active":"2024-03-11T19:51:56.713947+0000","last_peered":"2024-03-11T19:51:56.713947+0000","last_clean":"2024-03-11T19:51:56.713947+0000","last_became_active":"2024-03-11T19:49:54.717478+0000","last_became_peered":"2024-03-11T19:49:54.717478+0000","last_unstale":"2024-03-11T19:51:56.713947+0000","last_undegraded":"2024-03-11T19:51:56.713947+0000","last_fullsized":"2024-03-11T19:51:56.713947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:03.013940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526502+0000","last_change":"2024-03-11T19:49:55.236456+0000","last_active":"2024-03-11T19:51:56.526502+0000","last_peered":"2024-03-11T19:51:56.526502+0000","last_clean":"2024-03-11T19:51:56.526502+0000","last_became_active":"2024-03-11T19:49:55.236046+0000","last_became_peered":"2024-03-11T19:49:55.236046+0000","last_unstale":"2024-03-11T19:51:56.526502+0000","last_undegraded":"2024-03-11T19:51:56.526502+0000","last_fullsized":"2024-03-11T19:51:56.526502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:20:28.648600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522659+0000","last_change":"2024-03-11T19:50:00.468274+0000","last_active":"2024-03-11T19:51:56.522659+0000","last_peered":"2024-03-11T19:51:56.522659+0000","last_clean":"2024-03-11T19:51:56.522659+0000","last_became_active":"2024-03-11T19:50:00.467856+0000","last_became_peered":"2024-03-11T19:50:00.467856+0000","last_unstale":"2024-03-11T19:51:56.522659+0000","last_undegraded":"2024-03-11T19:51:56.522659+0000","last_fullsized":"2024-03-11T19:51:56.522659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:25.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527995+0000","last_change":"2024-03-11T19:49:54.731172+0000","last_active":"2024-03-11T19:51:56.527995+0000","last_peered":"2024-03-11T19:51:56.527995+0000","last_clean":"2024-03-11T19:51:56.527995+0000","last_became_active":"2024-03-11T19:49:54.727572+0000","last_became_peered":"2024-03-11T19:49:54.727572+0000","last_unstale":"2024-03-11T19:51:56.527995+0000","last_undegraded":"2024-03-11T19:51:56.527995+0000","last_fullsized":"2024-03-11T19:51:56.527995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:17.299017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540550+0000","last_change":"2024-03-11T19:49:55.228080+0000","last_active":"2024-03-11T19:51:56.540550+0000","last_peered":"2024-03-11T19:51:56.540550+0000","last_clean":"2024-03-11T19:51:56.540550+0000","last_became_active":"2024-03-11T19:49:55.227600+0000","last_became_peered":"2024-03-11T19:49:55.227600+0000","last_unstale":"2024-03-11T19:51:56.540550+0000","last_undegraded":"2024-03-11T19:51:56.540550+0000","last_fullsized":"2024-03-11T19:51:56.540550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:03.494513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522810+0000","last_change":"2024-03-11T19:50:00.468334+0000","last_active":"2024-03-11T19:51:56.522810+0000","last_peered":"2024-03-11T19:51:56.522810+0000","last_clean":"2024-03-11T19:51:56.522810+0000","last_became_active":"2024-03-11T19:50:00.467836+0000","last_became_peered":"2024-03-11T19:50:00.467836+0000","last_unstale":"2024-03-11T19:51:56.522810+0000","last_undegraded":"2024-03-11T19:51:56.522810+0000","last_fullsized":"2024-03-11T19:51:56.522810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:42.534042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709813+0000","last_change":"2024-03-11T19:50:55.480971+0000","last_active":"2024-03-11T19:51:56.709813+0000","last_peered":"2024-03-11T19:51:56.709813+0000","last_clean":"2024-03-11T19:51:56.709813+0000","last_became_active":"2024-03-11T19:50:55.474612+0000","last_became_peered":"2024-03-11T19:50:55.474612+0000","last_unstale":"2024-03-11T19:51:56.709813+0000","last_undegraded":"2024-03-11T19:51:56.709813+0000","last_fullsized":"2024-03-11T19:51:56.709813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:59:47.829059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709488+0000","last_change":"2024-03-11T19:49:55.209260+0000","last_active":"2024-03-11T19:51:56.709488+0000","last_peered":"2024-03-11T19:51:56.709488+0000","last_clean":"2024-03-11T19:51:56.709488+0000","last_became_active":"2024-03-11T19:49:55.208470+0000","last_became_peered":"2024-03-11T19:49:55.208470+0000","last_unstale":"2024-03-11T19:51:56.709488+0000","last_undegraded":"2024-03-11T19:51:56.709488+0000","last_fullsized":"2024-03-11T19:51:56.709488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:45:09.641403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522632+0000","last_change":"2024-03-11T19:50:00.461478+0000","last_active":"2024-03-11T19:51:56.522632+0000","last_peered":"2024-03-11T19:51:56.522632+0000","last_clean":"2024-03-11T19:51:56.522632+0000","last_became_active":"2024-03-11T19:50:00.461034+0000","last_became_peered":"2024-03-11T19:50:00.461034+0000","last_unstale":"2024-03-11T19:51:56.522632+0000","last_undegraded":"2024-03-11T19:51:56.522632+0000","last_fullsized":"2024-03-11T19:51:56.522632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:08.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522758+0000","last_change":"2024-03-11T19:49:54.711481+0000","last_active":"2024-03-11T19:51:56.522758+0000","last_peered":"2024-03-11T19:51:56.522758+0000","last_clean":"2024-03-11T19:51:56.522758+0000","last_became_active":"2024-03-11T19:49:54.711158+0000","last_became_peered":"2024-03-11T19:49:54.711158+0000","last_unstale":"2024-03-11T19:51:56.522758+0000","last_undegraded":"2024-03-11T19:51:56.522758+0000","last_fullsized":"2024-03-11T19:51:56.522758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:42:40.917970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596489+0000","last_change":"2024-03-11T19:49:55.214959+0000","last_active":"2024-03-11T19:51:56.596489+0000","last_peered":"2024-03-11T19:51:56.596489+0000","last_clean":"2024-03-11T19:51:56.596489+0000","last_became_active":"2024-03-11T19:49:55.214645+0000","last_became_peered":"2024-03-11T19:49:55.214645+0000","last_unstale":"2024-03-11T19:51:56.596489+0000","last_undegraded":"2024-03-11T19:51:56.596489+0000","last_fullsized":"2024-03-11T19:51:56.596489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:39:07.859775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709721+0000","last_change":"2024-03-11T19:50:00.453519+0000","last_active":"2024-03-11T19:51:56.709721+0000","last_peered":"2024-03-11T19:51:56.709721+0000","last_clean":"2024-03-11T19:51:56.709721+0000","last_became_active":"2024-03-11T19:50:00.452755+0000","last_became_peered":"2024-03-11T19:50:00.452755+0000","last_unstale":"2024-03-11T19:51:56.709721+0000","last_undegraded":"2024-03-11T19:51:56.709721+0000","last_fullsized":"2024-03-11T19:51:56.709721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:38:11.741412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:16.625381+0000","last_change":"2024-03-11T19:50:55.509345+0000","last_active":"2024-03-11T19:52:16.625381+0000","last_peered":"2024-03-11T19:52:16.625381+0000","last_clean":"2024-03-11T19:52:16.625381+0000","last_became_active":"2024-03-11T19:50:55.505792+0000","last_became_peered":"2024-03-11T19:50:55.505792+0000","last_unstale":"2024-03-11T19:52:16.625381+0000","last_undegraded":"2024-03-11T19:52:16.625381+0000","last_fullsized":"2024-03-11T19:52:16.625381+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:01:20.682572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522735+0000","last_change":"2024-03-11T19:49:55.228435+0000","last_active":"2024-03-11T19:51:56.522735+0000","last_peered":"2024-03-11T19:51:56.522735+0000","last_clean":"2024-03-11T19:51:56.522735+0000","last_became_active":"2024-03-11T19:49:55.228079+0000","last_became_peered":"2024-03-11T19:49:55.228079+0000","last_unstale":"2024-03-11T19:51:56.522735+0000","last_undegraded":"2024-03-11T19:51:56.522735+0000","last_fullsized":"2024-03-11T19:51:56.522735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:05:33.215656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709487+0000","last_change":"2024-03-11T19:50:00.478458+0000","last_active":"2024-03-11T19:51:56.709487+0000","last_peered":"2024-03-11T19:51:56.709487+0000","last_clean":"2024-03-11T19:51:56.709487+0000","last_became_active":"2024-03-11T19:50:00.478082+0000","last_became_peered":"2024-03-11T19:50:00.478082+0000","last_unstale":"2024-03-11T19:51:56.709487+0000","last_undegraded":"2024-03-11T19:51:56.709487+0000","last_fullsized":"2024-03-11T19:51:56.709487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:15:11.464270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532197+0000","last_change":"2024-03-11T19:49:54.720908+0000","last_active":"2024-03-11T19:51:56.532197+0000","last_peered":"2024-03-11T19:51:56.532197+0000","last_clean":"2024-03-11T19:51:56.532197+0000","last_became_active":"2024-03-11T19:49:54.720597+0000","last_became_peered":"2024-03-11T19:49:54.720597+0000","last_unstale":"2024-03-11T19:51:56.532197+0000","last_undegraded":"2024-03-11T19:51:56.532197+0000","last_fullsized":"2024-03-11T19:51:56.532197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:59:23.403911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596519+0000","last_change":"2024-03-11T19:49:55.234072+0000","last_active":"2024-03-11T19:51:56.596519+0000","last_peered":"2024-03-11T19:51:56.596519+0000","last_clean":"2024-03-11T19:51:56.596519+0000","last_became_active":"2024-03-11T19:49:55.233745+0000","last_became_peered":"2024-03-11T19:49:55.233745+0000","last_unstale":"2024-03-11T19:51:56.596519+0000","last_undegraded":"2024-03-11T19:51:56.596519+0000","last_fullsized":"2024-03-11T19:51:56.596519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:31.460323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709685+0000","last_change":"2024-03-11T19:50:00.474547+0000","last_active":"2024-03-11T19:51:56.709685+0000","last_peered":"2024-03-11T19:51:56.709685+0000","last_clean":"2024-03-11T19:51:56.709685+0000","last_became_active":"2024-03-11T19:50:00.474059+0000","last_became_peered":"2024-03-11T19:50:00.474059+0000","last_unstale":"2024-03-11T19:51:56.709685+0000","last_undegraded":"2024-03-11T19:51:56.709685+0000","last_fullsized":"2024-03-11T19:51:56.709685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:58:12.187159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.529497+0000","last_change":"2024-03-11T19:49:54.723936+0000","last_active":"2024-03-11T19:51:56.529497+0000","last_peered":"2024-03-11T19:51:56.529497+0000","last_clean":"2024-03-11T19:51:56.529497+0000","last_became_active":"2024-03-11T19:49:54.723140+0000","last_became_peered":"2024-03-11T19:49:54.723140+0000","last_unstale":"2024-03-11T19:51:56.529497+0000","last_undegraded":"2024-03-11T19:51:56.529497+0000","last_fullsized":"2024-03-11T19:51:56.529497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:33:34.566380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711388+0000","last_change":"2024-03-11T19:51:55.547947+0000","last_active":"2024-03-11T19:51:56.711388+0000","last_peered":"2024-03-11T19:51:56.711388+0000","last_clean":"2024-03-11T19:51:56.711388+0000","last_became_active":"2024-03-11T19:51:55.547500+0000","last_became_peered":"2024-03-11T19:51:55.547500+0000","last_unstale":"2024-03-11T19:51:56.711388+0000","last_undegraded":"2024-03-11T19:51:56.711388+0000","last_fullsized":"2024-03-11T19:51:56.711388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:44:01.043269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539927+0000","last_change":"2024-03-11T19:50:00.471915+0000","last_active":"2024-03-11T19:51:56.539927+0000","last_peered":"2024-03-11T19:51:56.539927+0000","last_clean":"2024-03-11T19:51:56.539927+0000","last_became_active":"2024-03-11T19:50:00.471409+0000","last_became_peered":"2024-03-11T19:50:00.471409+0000","last_unstale":"2024-03-11T19:51:56.539927+0000","last_undegraded":"2024-03-11T19:51:56.539927+0000","last_fullsized":"2024-03-11T19:51:56.539927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:17.372381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596062+0000","last_change":"2024-03-11T19:49:54.713564+0000","last_active":"2024-03-11T19:51:56.596062+0000","last_peered":"2024-03-11T19:51:56.596062+0000","last_clean":"2024-03-11T19:51:56.596062+0000","last_became_active":"2024-03-11T19:49:54.713267+0000","last_became_peered":"2024-03-11T19:49:54.713267+0000","last_unstale":"2024-03-11T19:51:56.596062+0000","last_undegraded":"2024-03-11T19:51:56.596062+0000","last_fullsized":"2024-03-11T19:51:56.596062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:48:59.284898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712060+0000","last_change":"2024-03-11T19:49:55.237909+0000","last_active":"2024-03-11T19:51:56.712060+0000","last_peered":"2024-03-11T19:51:56.712060+0000","last_clean":"2024-03-11T19:51:56.712060+0000","last_became_active":"2024-03-11T19:49:55.237592+0000","last_became_peered":"2024-03-11T19:49:55.237592+0000","last_unstale":"2024-03-11T19:51:56.712060+0000","last_undegraded":"2024-03-11T19:51:56.712060+0000","last_fullsized":"2024-03-11T19:51:56.712060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:46:34.327745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525570+0000","last_change":"2024-03-11T19:50:00.456896+0000","last_active":"2024-03-11T19:51:56.525570+0000","last_peered":"2024-03-11T19:51:56.525570+0000","last_clean":"2024-03-11T19:51:56.525570+0000","last_became_active":"2024-03-11T19:50:00.455780+0000","last_became_peered":"2024-03-11T19:50:00.455780+0000","last_unstale":"2024-03-11T19:51:56.525570+0000","last_undegraded":"2024-03-11T19:51:56.525570+0000","last_fullsized":"2024-03-11T19:51:56.525570+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:54:10.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":145,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:16.624780+0000","last_change":"2024-03-11T19:49:54.717969+0000","last_active":"2024-03-11T19:52:16.624780+0000","last_peered":"2024-03-11T19:52:16.624780+0000","last_clean":"2024-03-11T19:52:16.624780+0000","last_became_active":"2024-03-11T19:49:54.717646+0000","last_became_peered":"2024-03-11T19:49:54.717646+0000","last_unstale":"2024-03-11T19:52:16.624780+0000","last_undegraded":"2024-03-11T19:52:16.624780+0000","last_fullsized":"2024-03-11T19:52:16.624780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:28:26.327307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709914+0000","last_change":"2024-03-11T19:49:55.231319+0000","last_active":"2024-03-11T19:51:56.709914+0000","last_peered":"2024-03-11T19:51:56.709914+0000","last_clean":"2024-03-11T19:51:56.709914+0000","last_became_active":"2024-03-11T19:49:55.230966+0000","last_became_peered":"2024-03-11T19:49:55.230966+0000","last_unstale":"2024-03-11T19:51:56.709914+0000","last_undegraded":"2024-03-11T19:51:56.709914+0000","last_fullsized":"2024-03-11T19:51:56.709914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:46:40.871100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.711767+0000","last_change":"2024-03-11T19:50:00.476834+0000","last_active":"2024-03-11T19:51:56.711767+0000","last_peered":"2024-03-11T19:51:56.711767+0000","last_clean":"2024-03-11T19:51:56.711767+0000","last_became_active":"2024-03-11T19:50:00.473536+0000","last_became_peered":"2024-03-11T19:50:00.473536+0000","last_unstale":"2024-03-11T19:51:56.711767+0000","last_undegraded":"2024-03-11T19:51:56.711767+0000","last_fullsized":"2024-03-11T19:51:56.711767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:34.490972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709292+0000","last_change":"2024-03-11T19:49:54.717096+0000","last_active":"2024-03-11T19:51:56.709292+0000","last_peered":"2024-03-11T19:51:56.709292+0000","last_clean":"2024-03-11T19:51:56.709292+0000","last_became_active":"2024-03-11T19:49:54.716746+0000","last_became_peered":"2024-03-11T19:49:54.716746+0000","last_unstale":"2024-03-11T19:51:56.709292+0000","last_undegraded":"2024-03-11T19:51:56.709292+0000","last_fullsized":"2024-03-11T19:51:56.709292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:35:54.574514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540153+0000","last_change":"2024-03-11T19:51:55.547076+0000","last_active":"2024-03-11T19:51:56.540153+0000","last_peered":"2024-03-11T19:51:56.540153+0000","last_clean":"2024-03-11T19:51:56.540153+0000","last_became_active":"2024-03-11T19:51:55.546410+0000","last_became_peered":"2024-03-11T19:51:55.546410+0000","last_unstale":"2024-03-11T19:51:56.540153+0000","last_undegraded":"2024-03-11T19:51:56.540153+0000","last_fullsized":"2024-03-11T19:51:56.540153+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:43:21.121511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,8],"acting":[0,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522993+0000","last_change":"2024-03-11T19:50:00.475695+0000","last_active":"2024-03-11T19:51:56.522993+0000","last_peered":"2024-03-11T19:51:56.522993+0000","last_clean":"2024-03-11T19:51:56.522993+0000","last_became_active":"2024-03-11T19:50:00.475340+0000","last_became_peered":"2024-03-11T19:50:00.475340+0000","last_unstale":"2024-03-11T19:51:56.522993+0000","last_undegraded":"2024-03-11T19:51:56.522993+0000","last_fullsized":"2024-03-11T19:51:56.522993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:04:58.631491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596354+0000","last_change":"2024-03-11T19:49:54.721262+0000","last_active":"2024-03-11T19:51:56.596354+0000","last_peered":"2024-03-11T19:51:56.596354+0000","last_clean":"2024-03-11T19:51:56.596354+0000","last_became_active":"2024-03-11T19:49:54.720933+0000","last_became_peered":"2024-03-11T19:49:54.720933+0000","last_unstale":"2024-03-11T19:51:56.596354+0000","last_undegraded":"2024-03-11T19:51:56.596354+0000","last_fullsized":"2024-03-11T19:51:56.596354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:19.764134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709730+0000","last_change":"2024-03-11T19:49:55.217763+0000","last_active":"2024-03-11T19:51:56.709730+0000","last_peered":"2024-03-11T19:51:56.709730+0000","last_clean":"2024-03-11T19:51:56.709730+0000","last_became_active":"2024-03-11T19:49:55.217373+0000","last_became_peered":"2024-03-11T19:49:55.217373+0000","last_unstale":"2024-03-11T19:51:56.709730+0000","last_undegraded":"2024-03-11T19:51:56.709730+0000","last_fullsized":"2024-03-11T19:51:56.709730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:12.011970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532035+0000","last_change":"2024-03-11T19:50:55.475760+0000","last_active":"2024-03-11T19:51:56.532035+0000","last_peered":"2024-03-11T19:51:56.532035+0000","last_clean":"2024-03-11T19:51:56.532035+0000","last_became_active":"2024-03-11T19:50:55.475170+0000","last_became_peered":"2024-03-11T19:50:55.475170+0000","last_unstale":"2024-03-11T19:51:56.532035+0000","last_undegraded":"2024-03-11T19:51:56.532035+0000","last_fullsized":"2024-03-11T19:51:56.532035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:58.278212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"92'35","reported_seq":293,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:17.626387+0000","last_change":"2024-03-11T19:49:54.714426+0000","last_active":"2024-03-11T19:52:17.626387+0000","last_peered":"2024-03-11T19:52:17.626387+0000","last_clean":"2024-03-11T19:52:17.626387+0000","last_became_active":"2024-03-11T19:49:54.714024+0000","last_became_peered":"2024-03-11T19:49:54.714024+0000","last_unstale":"2024-03-11T19:52:17.626387+0000","last_undegraded":"2024-03-11T19:52:17.626387+0000","last_fullsized":"2024-03-11T19:52:17.626387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:52:30.071688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":173,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523168+0000","last_change":"2024-03-11T19:49:55.208701+0000","last_active":"2024-03-11T19:51:56.523168+0000","last_peered":"2024-03-11T19:51:56.523168+0000","last_clean":"2024-03-11T19:51:56.523168+0000","last_became_active":"2024-03-11T19:49:55.208383+0000","last_became_peered":"2024-03-11T19:49:55.208383+0000","last_unstale":"2024-03-11T19:51:56.523168+0000","last_undegraded":"2024-03-11T19:51:56.523168+0000","last_fullsized":"2024-03-11T19:51:56.523168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:07.999906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596828+0000","last_change":"2024-03-11T19:50:00.468464+0000","last_active":"2024-03-11T19:51:56.596828+0000","last_peered":"2024-03-11T19:51:56.596828+0000","last_clean":"2024-03-11T19:51:56.596828+0000","last_became_active":"2024-03-11T19:50:00.468064+0000","last_became_peered":"2024-03-11T19:50:00.468064+0000","last_unstale":"2024-03-11T19:51:56.596828+0000","last_undegraded":"2024-03-11T19:51:56.596828+0000","last_fullsized":"2024-03-11T19:51:56.596828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:02:46.167198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709090+0000","last_change":"2024-03-11T19:49:54.716089+0000","last_active":"2024-03-11T19:51:56.709090+0000","last_peered":"2024-03-11T19:51:56.709090+0000","last_clean":"2024-03-11T19:51:56.709090+0000","last_became_active":"2024-03-11T19:49:54.709644+0000","last_became_peered":"2024-03-11T19:49:54.709644+0000","last_unstale":"2024-03-11T19:51:56.709090+0000","last_undegraded":"2024-03-11T19:51:56.709090+0000","last_fullsized":"2024-03-11T19:51:56.709090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:53:23.775113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530863+0000","last_change":"2024-03-11T19:49:55.238980+0000","last_active":"2024-03-11T19:51:56.530863+0000","last_peered":"2024-03-11T19:51:56.530863+0000","last_clean":"2024-03-11T19:51:56.530863+0000","last_became_active":"2024-03-11T19:49:55.238632+0000","last_became_peered":"2024-03-11T19:49:55.238632+0000","last_unstale":"2024-03-11T19:51:56.530863+0000","last_undegraded":"2024-03-11T19:51:56.530863+0000","last_fullsized":"2024-03-11T19:51:56.530863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:35:50.957601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709442+0000","last_change":"2024-03-11T19:50:00.472854+0000","last_active":"2024-03-11T19:51:56.709442+0000","last_peered":"2024-03-11T19:51:56.709442+0000","last_clean":"2024-03-11T19:51:56.709442+0000","last_became_active":"2024-03-11T19:50:00.472490+0000","last_became_peered":"2024-03-11T19:50:00.472490+0000","last_unstale":"2024-03-11T19:51:56.709442+0000","last_undegraded":"2024-03-11T19:51:56.709442+0000","last_fullsized":"2024-03-11T19:51:56.709442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:46.697120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596410+0000","last_change":"2024-03-11T19:49:54.718392+0000","last_active":"2024-03-11T19:51:56.596410+0000","last_peered":"2024-03-11T19:51:56.596410+0000","last_clean":"2024-03-11T19:51:56.596410+0000","last_became_active":"2024-03-11T19:49:54.718042+0000","last_became_peered":"2024-03-11T19:49:54.718042+0000","last_unstale":"2024-03-11T19:51:56.596410+0000","last_undegraded":"2024-03-11T19:51:56.596410+0000","last_fullsized":"2024-03-11T19:51:56.596410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:33.761319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710129+0000","last_change":"2024-03-11T19:49:55.215381+0000","last_active":"2024-03-11T19:51:56.710129+0000","last_peered":"2024-03-11T19:51:56.710129+0000","last_clean":"2024-03-11T19:51:56.710129+0000","last_became_active":"2024-03-11T19:49:55.214921+0000","last_became_peered":"2024-03-11T19:49:55.214921+0000","last_unstale":"2024-03-11T19:51:56.710129+0000","last_undegraded":"2024-03-11T19:51:56.710129+0000","last_fullsized":"2024-03-11T19:51:56.710129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:00:18.224739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713445+0000","last_change":"2024-03-11T19:50:00.479243+0000","last_active":"2024-03-11T19:51:56.713445+0000","last_peered":"2024-03-11T19:51:56.713445+0000","last_clean":"2024-03-11T19:51:56.713445+0000","last_became_active":"2024-03-11T19:50:00.474867+0000","last_became_peered":"2024-03-11T19:50:00.474867+0000","last_unstale":"2024-03-11T19:51:56.713445+0000","last_undegraded":"2024-03-11T19:51:56.713445+0000","last_fullsized":"2024-03-11T19:51:56.713445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:43:17.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":155,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:18.687748+0000","last_change":"2024-03-11T19:49:54.709604+0000","last_active":"2024-03-11T19:52:18.687748+0000","last_peered":"2024-03-11T19:52:18.687748+0000","last_clean":"2024-03-11T19:52:18.687748+0000","last_became_active":"2024-03-11T19:49:54.709275+0000","last_became_peered":"2024-03-11T19:49:54.709275+0000","last_unstale":"2024-03-11T19:52:18.687748+0000","last_undegraded":"2024-03-11T19:52:18.687748+0000","last_fullsized":"2024-03-11T19:52:18.687748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:46:25.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708945+0000","last_change":"2024-03-11T19:49:55.209844+0000","last_active":"2024-03-11T19:51:56.708945+0000","last_peered":"2024-03-11T19:51:56.708945+0000","last_clean":"2024-03-11T19:51:56.708945+0000","last_became_active":"2024-03-11T19:49:55.209538+0000","last_became_peered":"2024-03-11T19:49:55.209538+0000","last_unstale":"2024-03-11T19:51:56.708945+0000","last_undegraded":"2024-03-11T19:51:56.708945+0000","last_fullsized":"2024-03-11T19:51:56.708945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.716907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531921+0000","last_change":"2024-03-11T19:50:00.474353+0000","last_active":"2024-03-11T19:51:56.531921+0000","last_peered":"2024-03-11T19:51:56.531921+0000","last_clean":"2024-03-11T19:51:56.531921+0000","last_became_active":"2024-03-11T19:50:00.473938+0000","last_became_peered":"2024-03-11T19:50:00.473938+0000","last_unstale":"2024-03-11T19:51:56.531921+0000","last_undegraded":"2024-03-11T19:51:56.531921+0000","last_fullsized":"2024-03-11T19:51:56.531921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:22:54.522454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522616+0000","last_change":"2024-03-11T19:50:55.807772+0000","last_active":"2024-03-11T19:51:56.522616+0000","last_peered":"2024-03-11T19:51:56.522616+0000","last_clean":"2024-03-11T19:51:56.522616+0000","last_became_active":"2024-03-11T19:50:55.794019+0000","last_became_peered":"2024-03-11T19:50:55.794019+0000","last_unstale":"2024-03-11T19:51:56.522616+0000","last_undegraded":"2024-03-11T19:51:56.522616+0000","last_fullsized":"2024-03-11T19:51:56.522616+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:00.170150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596224+0000","last_change":"2024-03-11T19:50:55.793868+0000","last_active":"2024-03-11T19:51:56.596224+0000","last_peered":"2024-03-11T19:51:56.596224+0000","last_clean":"2024-03-11T19:51:56.596224+0000","last_became_active":"2024-03-11T19:50:55.793299+0000","last_became_peered":"2024-03-11T19:50:55.793299+0000","last_unstale":"2024-03-11T19:51:56.596224+0000","last_undegraded":"2024-03-11T19:51:56.596224+0000","last_fullsized":"2024-03-11T19:51:56.596224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:40:42.039449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596213+0000","last_change":"2024-03-11T19:50:00.462535+0000","last_active":"2024-03-11T19:51:56.596213+0000","last_peered":"2024-03-11T19:51:56.596213+0000","last_clean":"2024-03-11T19:51:56.596213+0000","last_became_active":"2024-03-11T19:50:00.462175+0000","last_became_peered":"2024-03-11T19:50:00.462175+0000","last_unstale":"2024-03-11T19:51:56.596213+0000","last_undegraded":"2024-03-11T19:51:56.596213+0000","last_fullsized":"2024-03-11T19:51:56.596213+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:36.769828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523392+0000","last_change":"2024-03-11T19:49:54.708268+0000","last_active":"2024-03-11T19:51:56.523392+0000","last_peered":"2024-03-11T19:51:56.523392+0000","last_clean":"2024-03-11T19:51:56.523392+0000","last_became_active":"2024-03-11T19:49:54.707911+0000","last_became_peered":"2024-03-11T19:49:54.707911+0000","last_unstale":"2024-03-11T19:51:56.523392+0000","last_undegraded":"2024-03-11T19:51:56.523392+0000","last_fullsized":"2024-03-11T19:51:56.523392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:02:05.488768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709639+0000","last_change":"2024-03-11T19:49:55.209452+0000","last_active":"2024-03-11T19:51:56.709639+0000","last_peered":"2024-03-11T19:51:56.709639+0000","last_clean":"2024-03-11T19:51:56.709639+0000","last_became_active":"2024-03-11T19:49:55.209093+0000","last_became_peered":"2024-03-11T19:49:55.209093+0000","last_unstale":"2024-03-11T19:51:56.709639+0000","last_undegraded":"2024-03-11T19:51:56.709639+0000","last_fullsized":"2024-03-11T19:51:56.709639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:44:59.349415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522834+0000","last_change":"2024-03-11T19:50:00.581147+0000","last_active":"2024-03-11T19:51:56.522834+0000","last_peered":"2024-03-11T19:51:56.522834+0000","last_clean":"2024-03-11T19:51:56.522834+0000","last_became_active":"2024-03-11T19:50:00.580692+0000","last_became_peered":"2024-03-11T19:50:00.580692+0000","last_unstale":"2024-03-11T19:51:56.522834+0000","last_undegraded":"2024-03-11T19:51:56.522834+0000","last_fullsized":"2024-03-11T19:51:56.522834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:58.732795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.712674+0000","last_change":"2024-03-11T19:49:54.717278+0000","last_active":"2024-03-11T19:51:56.712674+0000","last_peered":"2024-03-11T19:51:56.712674+0000","last_clean":"2024-03-11T19:51:56.712674+0000","last_became_active":"2024-03-11T19:49:54.716823+0000","last_became_peered":"2024-03-11T19:49:54.716823+0000","last_unstale":"2024-03-11T19:51:56.712674+0000","last_undegraded":"2024-03-11T19:51:56.712674+0000","last_fullsized":"2024-03-11T19:51:56.712674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:51:13.432554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532585+0000","last_change":"2024-03-11T19:49:55.234533+0000","last_active":"2024-03-11T19:51:56.532585+0000","last_peered":"2024-03-11T19:51:56.532585+0000","last_clean":"2024-03-11T19:51:56.532585+0000","last_became_active":"2024-03-11T19:49:55.234255+0000","last_became_peered":"2024-03-11T19:49:55.234255+0000","last_unstale":"2024-03-11T19:51:56.532585+0000","last_undegraded":"2024-03-11T19:51:56.532585+0000","last_fullsized":"2024-03-11T19:51:56.532585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:10:19.560036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531529+0000","last_change":"2024-03-11T19:50:00.479181+0000","last_active":"2024-03-11T19:51:56.531529+0000","last_peered":"2024-03-11T19:51:56.531529+0000","last_clean":"2024-03-11T19:51:56.531529+0000","last_became_active":"2024-03-11T19:50:00.477902+0000","last_became_peered":"2024-03-11T19:50:00.477902+0000","last_unstale":"2024-03-11T19:51:56.531529+0000","last_undegraded":"2024-03-11T19:51:56.531529+0000","last_fullsized":"2024-03-11T19:51:56.531529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:02:25.504449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709519+0000","last_change":"2024-03-11T19:49:54.723029+0000","last_active":"2024-03-11T19:51:56.709519+0000","last_peered":"2024-03-11T19:51:56.709519+0000","last_clean":"2024-03-11T19:51:56.709519+0000","last_became_active":"2024-03-11T19:49:54.722691+0000","last_became_peered":"2024-03-11T19:49:54.722691+0000","last_unstale":"2024-03-11T19:51:56.709519+0000","last_undegraded":"2024-03-11T19:51:56.709519+0000","last_fullsized":"2024-03-11T19:51:56.709519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:31:19.172373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532690+0000","last_change":"2024-03-11T19:49:55.222179+0000","last_active":"2024-03-11T19:51:56.532690+0000","last_peered":"2024-03-11T19:51:56.532690+0000","last_clean":"2024-03-11T19:51:56.532690+0000","last_became_active":"2024-03-11T19:49:55.221818+0000","last_became_peered":"2024-03-11T19:49:55.221818+0000","last_unstale":"2024-03-11T19:51:56.532690+0000","last_undegraded":"2024-03-11T19:51:56.532690+0000","last_fullsized":"2024-03-11T19:51:56.532690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:23:36.172327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532450+0000","last_change":"2024-03-11T19:50:00.582017+0000","last_active":"2024-03-11T19:51:56.532450+0000","last_peered":"2024-03-11T19:51:56.532450+0000","last_clean":"2024-03-11T19:51:56.532450+0000","last_became_active":"2024-03-11T19:50:00.580861+0000","last_became_peered":"2024-03-11T19:50:00.580861+0000","last_unstale":"2024-03-11T19:51:56.532450+0000","last_undegraded":"2024-03-11T19:51:56.532450+0000","last_fullsized":"2024-03-11T19:51:56.532450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:41.311132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596553+0000","last_change":"2024-03-11T19:49:55.212690+0000","last_active":"2024-03-11T19:51:56.596553+0000","last_peered":"2024-03-11T19:51:56.596553+0000","last_clean":"2024-03-11T19:51:56.596553+0000","last_became_active":"2024-03-11T19:49:55.212390+0000","last_became_peered":"2024-03-11T19:49:55.212390+0000","last_unstale":"2024-03-11T19:51:56.596553+0000","last_undegraded":"2024-03-11T19:51:56.596553+0000","last_fullsized":"2024-03-11T19:51:56.596553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:01.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":150,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:52:17.684632+0000","last_change":"2024-03-11T19:49:54.724305+0000","last_active":"2024-03-11T19:52:17.684632+0000","last_peered":"2024-03-11T19:52:17.684632+0000","last_clean":"2024-03-11T19:52:17.684632+0000","last_became_active":"2024-03-11T19:49:54.723620+0000","last_became_peered":"2024-03-11T19:49:54.723620+0000","last_unstale":"2024-03-11T19:52:17.684632+0000","last_undegraded":"2024-03-11T19:52:17.684632+0000","last_fullsized":"2024-03-11T19:52:17.684632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:33:28.818166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709620+0000","last_change":"2024-03-11T19:50:55.488155+0000","last_active":"2024-03-11T19:51:56.709620+0000","last_peered":"2024-03-11T19:51:56.709620+0000","last_clean":"2024-03-11T19:51:56.709620+0000","last_became_active":"2024-03-11T19:50:55.477254+0000","last_became_peered":"2024-03-11T19:50:55.477254+0000","last_unstale":"2024-03-11T19:51:56.709620+0000","last_undegraded":"2024-03-11T19:51:56.709620+0000","last_fullsized":"2024-03-11T19:51:56.709620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:05.827200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527420+0000","last_change":"2024-03-11T19:49:55.236049+0000","last_active":"2024-03-11T19:51:56.527420+0000","last_peered":"2024-03-11T19:51:56.527420+0000","last_clean":"2024-03-11T19:51:56.527420+0000","last_became_active":"2024-03-11T19:49:55.230242+0000","last_became_peered":"2024-03-11T19:49:55.230242+0000","last_unstale":"2024-03-11T19:51:56.527420+0000","last_undegraded":"2024-03-11T19:51:56.527420+0000","last_fullsized":"2024-03-11T19:51:56.527420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:29:31.182086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522985+0000","last_change":"2024-03-11T19:49:54.713095+0000","last_active":"2024-03-11T19:51:56.522985+0000","last_peered":"2024-03-11T19:51:56.522985+0000","last_clean":"2024-03-11T19:51:56.522985+0000","last_became_active":"2024-03-11T19:49:54.712768+0000","last_became_peered":"2024-03-11T19:49:54.712768+0000","last_unstale":"2024-03-11T19:51:56.522985+0000","last_undegraded":"2024-03-11T19:51:56.522985+0000","last_fullsized":"2024-03-11T19:51:56.522985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:23.061148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708751+0000","last_change":"2024-03-11T19:48:35.390464+0000","last_active":"2024-03-11T19:51:56.708751+0000","last_peered":"2024-03-11T19:51:56.708751+0000","last_clean":"2024-03-11T19:51:56.708751+0000","last_became_active":"2024-03-11T19:48:35.390020+0000","last_became_peered":"2024-03-11T19:48:35.390020+0000","last_unstale":"2024-03-11T19:51:56.708751+0000","last_undegraded":"2024-03-11T19:51:56.708751+0000","last_fullsized":"2024-03-11T19:51:56.708751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:44:40.865519+0000","last_clean_scrub_stamp":"2024-03-11T19:44:40.865519+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:41:05.837968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596856+0000","last_change":"2024-03-11T19:50:00.581668+0000","last_active":"2024-03-11T19:51:56.596856+0000","last_peered":"2024-03-11T19:51:56.596856+0000","last_clean":"2024-03-11T19:51:56.596856+0000","last_became_active":"2024-03-11T19:50:00.580940+0000","last_became_peered":"2024-03-11T19:50:00.580940+0000","last_unstale":"2024-03-11T19:51:56.596856+0000","last_undegraded":"2024-03-11T19:51:56.596856+0000","last_fullsized":"2024-03-11T19:51:56.596856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:08:46.847785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.528427+0000","last_change":"2024-03-11T19:49:55.218584+0000","last_active":"2024-03-11T19:51:56.528427+0000","last_peered":"2024-03-11T19:51:56.528427+0000","last_clean":"2024-03-11T19:51:56.528427+0000","last_became_active":"2024-03-11T19:49:55.217827+0000","last_became_peered":"2024-03-11T19:49:55.217827+0000","last_unstale":"2024-03-11T19:51:56.528427+0000","last_undegraded":"2024-03-11T19:51:56.528427+0000","last_fullsized":"2024-03-11T19:51:56.528427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:44:56.810613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708807+0000","last_change":"2024-03-11T19:49:54.711611+0000","last_active":"2024-03-11T19:51:56.708807+0000","last_peered":"2024-03-11T19:51:56.708807+0000","last_clean":"2024-03-11T19:51:56.708807+0000","last_became_active":"2024-03-11T19:49:54.711316+0000","last_became_peered":"2024-03-11T19:49:54.711316+0000","last_unstale":"2024-03-11T19:51:56.708807+0000","last_undegraded":"2024-03-11T19:51:56.708807+0000","last_fullsized":"2024-03-11T19:51:56.708807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:28.746000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709363+0000","last_change":"2024-03-11T19:50:55.474853+0000","last_active":"2024-03-11T19:51:56.709363+0000","last_peered":"2024-03-11T19:51:56.709363+0000","last_clean":"2024-03-11T19:51:56.709363+0000","last_became_active":"2024-03-11T19:50:55.474379+0000","last_became_peered":"2024-03-11T19:50:55.474379+0000","last_unstale":"2024-03-11T19:51:56.709363+0000","last_undegraded":"2024-03-11T19:51:56.709363+0000","last_fullsized":"2024-03-11T19:51:56.709363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:41:43.691094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709662+0000","last_change":"2024-03-11T19:50:55.491848+0000","last_active":"2024-03-11T19:51:56.709662+0000","last_peered":"2024-03-11T19:51:56.709662+0000","last_clean":"2024-03-11T19:51:56.709662+0000","last_became_active":"2024-03-11T19:50:55.485786+0000","last_became_peered":"2024-03-11T19:50:55.485786+0000","last_unstale":"2024-03-11T19:51:56.709662+0000","last_undegraded":"2024-03-11T19:51:56.709662+0000","last_fullsized":"2024-03-11T19:51:56.709662+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:24:41.406176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.532724+0000","last_change":"2024-03-11T19:49:55.226630+0000","last_active":"2024-03-11T19:51:56.532724+0000","last_peered":"2024-03-11T19:51:56.532724+0000","last_clean":"2024-03-11T19:51:56.532724+0000","last_became_active":"2024-03-11T19:49:55.226305+0000","last_became_peered":"2024-03-11T19:49:55.226305+0000","last_unstale":"2024-03-11T19:51:56.532724+0000","last_undegraded":"2024-03-11T19:51:56.532724+0000","last_fullsized":"2024-03-11T19:51:56.532724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:46:42.818615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525309+0000","last_change":"2024-03-11T19:50:00.581652+0000","last_active":"2024-03-11T19:51:56.525309+0000","last_peered":"2024-03-11T19:51:56.525309+0000","last_clean":"2024-03-11T19:51:56.525309+0000","last_became_active":"2024-03-11T19:50:00.580853+0000","last_became_peered":"2024-03-11T19:50:00.580853+0000","last_unstale":"2024-03-11T19:51:56.525309+0000","last_undegraded":"2024-03-11T19:51:56.525309+0000","last_fullsized":"2024-03-11T19:51:56.525309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:27:10.922232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713268+0000","last_change":"2024-03-11T19:49:54.723593+0000","last_active":"2024-03-11T19:51:56.713268+0000","last_peered":"2024-03-11T19:51:56.713268+0000","last_clean":"2024-03-11T19:51:56.713268+0000","last_became_active":"2024-03-11T19:49:54.723200+0000","last_became_peered":"2024-03-11T19:49:54.723200+0000","last_unstale":"2024-03-11T19:51:56.713268+0000","last_undegraded":"2024-03-11T19:51:56.713268+0000","last_fullsized":"2024-03-11T19:51:56.713268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:14:24.136156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526748+0000","last_change":"2024-03-11T19:49:55.232750+0000","last_active":"2024-03-11T19:51:56.526748+0000","last_peered":"2024-03-11T19:51:56.526748+0000","last_clean":"2024-03-11T19:51:56.526748+0000","last_became_active":"2024-03-11T19:49:55.229885+0000","last_became_peered":"2024-03-11T19:49:55.229885+0000","last_unstale":"2024-03-11T19:51:56.526748+0000","last_undegraded":"2024-03-11T19:51:56.526748+0000","last_fullsized":"2024-03-11T19:51:56.526748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:21.607271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708772+0000","last_change":"2024-03-11T19:50:00.451867+0000","last_active":"2024-03-11T19:51:56.708772+0000","last_peered":"2024-03-11T19:51:56.708772+0000","last_clean":"2024-03-11T19:51:56.708772+0000","last_became_active":"2024-03-11T19:50:00.451460+0000","last_became_peered":"2024-03-11T19:50:00.451460+0000","last_unstale":"2024-03-11T19:51:56.708772+0000","last_undegraded":"2024-03-11T19:51:56.708772+0000","last_fullsized":"2024-03-11T19:51:56.708772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:18.909886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523380+0000","last_change":"2024-03-11T19:49:54.722953+0000","last_active":"2024-03-11T19:51:56.523380+0000","last_peered":"2024-03-11T19:51:56.523380+0000","last_clean":"2024-03-11T19:51:56.523380+0000","last_became_active":"2024-03-11T19:49:54.722610+0000","last_became_peered":"2024-03-11T19:49:54.722610+0000","last_unstale":"2024-03-11T19:51:56.523380+0000","last_undegraded":"2024-03-11T19:51:56.523380+0000","last_fullsized":"2024-03-11T19:51:56.523380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:21:58.553309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.540214+0000","last_change":"2024-03-11T19:49:55.234334+0000","last_active":"2024-03-11T19:51:56.540214+0000","last_peered":"2024-03-11T19:51:56.540214+0000","last_clean":"2024-03-11T19:51:56.540214+0000","last_became_active":"2024-03-11T19:49:55.234016+0000","last_became_peered":"2024-03-11T19:49:55.234016+0000","last_unstale":"2024-03-11T19:51:56.540214+0000","last_undegraded":"2024-03-11T19:51:56.540214+0000","last_fullsized":"2024-03-11T19:51:56.540214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:23:11.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523264+0000","last_change":"2024-03-11T19:50:00.581570+0000","last_active":"2024-03-11T19:51:56.523264+0000","last_peered":"2024-03-11T19:51:56.523264+0000","last_clean":"2024-03-11T19:51:56.523264+0000","last_became_active":"2024-03-11T19:50:00.580760+0000","last_became_peered":"2024-03-11T19:50:00.580760+0000","last_unstale":"2024-03-11T19:51:56.523264+0000","last_undegraded":"2024-03-11T19:51:56.523264+0000","last_fullsized":"2024-03-11T19:51:56.523264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:05:42.910365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709129+0000","last_change":"2024-03-11T19:49:54.709217+0000","last_active":"2024-03-11T19:51:56.709129+0000","last_peered":"2024-03-11T19:51:56.709129+0000","last_clean":"2024-03-11T19:51:56.709129+0000","last_became_active":"2024-03-11T19:49:54.708865+0000","last_became_peered":"2024-03-11T19:49:54.708865+0000","last_unstale":"2024-03-11T19:51:56.709129+0000","last_undegraded":"2024-03-11T19:51:56.709129+0000","last_fullsized":"2024-03-11T19:51:56.709129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:14:51.268069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522893+0000","last_change":"2024-03-11T19:49:55.208602+0000","last_active":"2024-03-11T19:51:56.522893+0000","last_peered":"2024-03-11T19:51:56.522893+0000","last_clean":"2024-03-11T19:51:56.522893+0000","last_became_active":"2024-03-11T19:49:55.208233+0000","last_became_peered":"2024-03-11T19:49:55.208233+0000","last_unstale":"2024-03-11T19:51:56.522893+0000","last_undegraded":"2024-03-11T19:51:56.522893+0000","last_fullsized":"2024-03-11T19:51:56.522893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:12:26.048100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.713854+0000","last_change":"2024-03-11T19:50:00.466844+0000","last_active":"2024-03-11T19:51:56.713854+0000","last_peered":"2024-03-11T19:51:56.713854+0000","last_clean":"2024-03-11T19:51:56.713854+0000","last_became_active":"2024-03-11T19:50:00.466332+0000","last_became_peered":"2024-03-11T19:50:00.466332+0000","last_unstale":"2024-03-11T19:51:56.713854+0000","last_undegraded":"2024-03-11T19:51:56.713854+0000","last_fullsized":"2024-03-11T19:51:56.713854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:23:10.023800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525952+0000","last_change":"2024-03-11T19:49:54.722147+0000","last_active":"2024-03-11T19:51:56.525952+0000","last_peered":"2024-03-11T19:51:56.525952+0000","last_clean":"2024-03-11T19:51:56.525952+0000","last_became_active":"2024-03-11T19:49:54.721801+0000","last_became_peered":"2024-03-11T19:49:54.721801+0000","last_unstale":"2024-03-11T19:51:56.525952+0000","last_undegraded":"2024-03-11T19:51:56.525952+0000","last_fullsized":"2024-03-11T19:51:56.525952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:41:02.705640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709572+0000","last_change":"2024-03-11T19:49:55.222125+0000","last_active":"2024-03-11T19:51:56.709572+0000","last_peered":"2024-03-11T19:51:56.709572+0000","last_clean":"2024-03-11T19:51:56.709572+0000","last_became_active":"2024-03-11T19:49:55.221791+0000","last_became_peered":"2024-03-11T19:49:55.221791+0000","last_unstale":"2024-03-11T19:51:56.709572+0000","last_undegraded":"2024-03-11T19:51:56.709572+0000","last_fullsized":"2024-03-11T19:51:56.709572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:18:34.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523061+0000","last_change":"2024-03-11T19:50:00.477967+0000","last_active":"2024-03-11T19:51:56.523061+0000","last_peered":"2024-03-11T19:51:56.523061+0000","last_clean":"2024-03-11T19:51:56.523061+0000","last_became_active":"2024-03-11T19:50:00.477407+0000","last_became_peered":"2024-03-11T19:50:00.477407+0000","last_unstale":"2024-03-11T19:51:56.523061+0000","last_undegraded":"2024-03-11T19:51:56.523061+0000","last_fullsized":"2024-03-11T19:51:56.523061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:02:48.337650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708920+0000","last_change":"2024-03-11T19:49:54.712872+0000","last_active":"2024-03-11T19:51:56.708920+0000","last_peered":"2024-03-11T19:51:56.708920+0000","last_clean":"2024-03-11T19:51:56.708920+0000","last_became_active":"2024-03-11T19:49:54.710508+0000","last_became_peered":"2024-03-11T19:49:54.710508+0000","last_unstale":"2024-03-11T19:51:56.708920+0000","last_undegraded":"2024-03-11T19:51:56.708920+0000","last_fullsized":"2024-03-11T19:51:56.708920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:51:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.525706+0000","last_change":"2024-03-11T19:49:55.233762+0000","last_active":"2024-03-11T19:51:56.525706+0000","last_peered":"2024-03-11T19:51:56.525706+0000","last_clean":"2024-03-11T19:51:56.525706+0000","last_became_active":"2024-03-11T19:49:55.232870+0000","last_became_peered":"2024-03-11T19:49:55.232870+0000","last_unstale":"2024-03-11T19:51:56.525706+0000","last_undegraded":"2024-03-11T19:51:56.525706+0000","last_fullsized":"2024-03-11T19:51:56.525706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:30.987225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.523041+0000","last_change":"2024-03-11T19:50:00.581031+0000","last_active":"2024-03-11T19:51:56.523041+0000","last_peered":"2024-03-11T19:51:56.523041+0000","last_clean":"2024-03-11T19:51:56.523041+0000","last_became_active":"2024-03-11T19:50:00.580630+0000","last_became_peered":"2024-03-11T19:50:00.580630+0000","last_unstale":"2024-03-11T19:51:56.523041+0000","last_undegraded":"2024-03-11T19:51:56.523041+0000","last_fullsized":"2024-03-11T19:51:56.523041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:47.732073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.522773+0000","last_change":"2024-03-11T19:49:54.707684+0000","last_active":"2024-03-11T19:51:56.522773+0000","last_peered":"2024-03-11T19:51:56.522773+0000","last_clean":"2024-03-11T19:51:56.522773+0000","last_became_active":"2024-03-11T19:49:54.707343+0000","last_became_peered":"2024-03-11T19:49:54.707343+0000","last_unstale":"2024-03-11T19:51:56.522773+0000","last_undegraded":"2024-03-11T19:51:56.522773+0000","last_fullsized":"2024-03-11T19:51:56.522773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:48:22.111449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709598+0000","last_change":"2024-03-11T19:49:55.219886+0000","last_active":"2024-03-11T19:51:56.709598+0000","last_peered":"2024-03-11T19:51:56.709598+0000","last_clean":"2024-03-11T19:51:56.709598+0000","last_became_active":"2024-03-11T19:49:55.219555+0000","last_became_peered":"2024-03-11T19:49:55.219555+0000","last_unstale":"2024-03-11T19:51:56.709598+0000","last_undegraded":"2024-03-11T19:51:56.709598+0000","last_fullsized":"2024-03-11T19:51:56.709598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:14:47.568309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708895+0000","last_change":"2024-03-11T19:50:00.460662+0000","last_active":"2024-03-11T19:51:56.708895+0000","last_peered":"2024-03-11T19:51:56.708895+0000","last_clean":"2024-03-11T19:51:56.708895+0000","last_became_active":"2024-03-11T19:50:00.460201+0000","last_became_peered":"2024-03-11T19:50:00.460201+0000","last_unstale":"2024-03-11T19:51:56.708895+0000","last_undegraded":"2024-03-11T19:51:56.708895+0000","last_fullsized":"2024-03-11T19:51:56.708895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:14:27.928483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709432+0000","last_change":"2024-03-11T19:49:54.712479+0000","last_active":"2024-03-11T19:51:56.709432+0000","last_peered":"2024-03-11T19:51:56.709432+0000","last_clean":"2024-03-11T19:51:56.709432+0000","last_became_active":"2024-03-11T19:49:54.710600+0000","last_became_peered":"2024-03-11T19:49:54.710600+0000","last_unstale":"2024-03-11T19:51:56.709432+0000","last_undegraded":"2024-03-11T19:51:56.709432+0000","last_fullsized":"2024-03-11T19:51:56.709432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:14:44.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.710064+0000","last_change":"2024-03-11T19:49:55.208414+0000","last_active":"2024-03-11T19:51:56.710064+0000","last_peered":"2024-03-11T19:51:56.710064+0000","last_clean":"2024-03-11T19:51:56.710064+0000","last_became_active":"2024-03-11T19:49:55.208112+0000","last_became_peered":"2024-03-11T19:49:55.208112+0000","last_unstale":"2024-03-11T19:51:56.710064+0000","last_undegraded":"2024-03-11T19:51:56.710064+0000","last_fullsized":"2024-03-11T19:51:56.710064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:55.348473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.538659+0000","last_change":"2024-03-11T19:50:00.476898+0000","last_active":"2024-03-11T19:51:56.538659+0000","last_peered":"2024-03-11T19:51:56.538659+0000","last_clean":"2024-03-11T19:51:56.538659+0000","last_became_active":"2024-03-11T19:50:00.475296+0000","last_became_peered":"2024-03-11T19:50:00.475296+0000","last_unstale":"2024-03-11T19:51:56.538659+0000","last_undegraded":"2024-03-11T19:51:56.538659+0000","last_fullsized":"2024-03-11T19:51:56.538659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:42:11.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596757+0000","last_change":"2024-03-11T19:49:54.719397+0000","last_active":"2024-03-11T19:51:56.596757+0000","last_peered":"2024-03-11T19:51:56.596757+0000","last_clean":"2024-03-11T19:51:56.596757+0000","last_became_active":"2024-03-11T19:49:54.719090+0000","last_became_peered":"2024-03-11T19:49:54.719090+0000","last_unstale":"2024-03-11T19:51:56.596757+0000","last_undegraded":"2024-03-11T19:51:56.596757+0000","last_fullsized":"2024-03-11T19:51:56.596757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:28:20.886503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709707+0000","last_change":"2024-03-11T19:49:55.230508+0000","last_active":"2024-03-11T19:51:56.709707+0000","last_peered":"2024-03-11T19:51:56.709707+0000","last_clean":"2024-03-11T19:51:56.709707+0000","last_became_active":"2024-03-11T19:49:55.230173+0000","last_became_peered":"2024-03-11T19:49:55.230173+0000","last_unstale":"2024-03-11T19:51:56.709707+0000","last_undegraded":"2024-03-11T19:51:56.709707+0000","last_fullsized":"2024-03-11T19:51:56.709707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:56:09.802806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.595995+0000","last_change":"2024-03-11T19:50:00.450827+0000","last_active":"2024-03-11T19:51:56.595995+0000","last_peered":"2024-03-11T19:51:56.595995+0000","last_clean":"2024-03-11T19:51:56.595995+0000","last_became_active":"2024-03-11T19:50:00.450455+0000","last_became_peered":"2024-03-11T19:50:00.450455+0000","last_unstale":"2024-03-11T19:51:56.595995+0000","last_undegraded":"2024-03-11T19:51:56.595995+0000","last_fullsized":"2024-03-11T19:51:56.595995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:59.404785+0000","last_clean_scrub_stamp":"2024-03-11T19:49:59.404785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:52.036177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.531929+0000","last_change":"2024-03-11T19:49:54.729341+0000","last_active":"2024-03-11T19:51:56.531929+0000","last_peered":"2024-03-11T19:51:56.531929+0000","last_clean":"2024-03-11T19:51:56.531929+0000","last_became_active":"2024-03-11T19:49:54.727426+0000","last_became_peered":"2024-03-11T19:49:54.727426+0000","last_unstale":"2024-03-11T19:51:56.531929+0000","last_undegraded":"2024-03-11T19:51:56.531929+0000","last_fullsized":"2024-03-11T19:51:56.531929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:53.673849+0000","last_clean_scrub_stamp":"2024-03-11T19:49:53.673849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:34.026616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709750+0000","last_change":"2024-03-11T19:49:55.211098+0000","last_active":"2024-03-11T19:51:56.709750+0000","last_peered":"2024-03-11T19:51:56.709750+0000","last_clean":"2024-03-11T19:51:56.709750+0000","last_became_active":"2024-03-11T19:49:55.210764+0000","last_became_peered":"2024-03-11T19:49:55.210764+0000","last_unstale":"2024-03-11T19:51:56.709750+0000","last_undegraded":"2024-03-11T19:51:56.709750+0000","last_fullsized":"2024-03-11T19:51:56.709750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:36:54.027632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709836+0000","last_change":"2024-03-11T19:49:55.211748+0000","last_active":"2024-03-11T19:51:56.709836+0000","last_peered":"2024-03-11T19:51:56.709836+0000","last_clean":"2024-03-11T19:51:56.709836+0000","last_became_active":"2024-03-11T19:49:55.211430+0000","last_became_peered":"2024-03-11T19:49:55.211430+0000","last_unstale":"2024-03-11T19:51:56.709836+0000","last_undegraded":"2024-03-11T19:51:56.709836+0000","last_fullsized":"2024-03-11T19:51:56.709836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:07:39.694437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708988+0000","last_change":"2024-03-11T19:49:55.215788+0000","last_active":"2024-03-11T19:51:56.708988+0000","last_peered":"2024-03-11T19:51:56.708988+0000","last_clean":"2024-03-11T19:51:56.708988+0000","last_became_active":"2024-03-11T19:49:55.215443+0000","last_became_peered":"2024-03-11T19:49:55.215443+0000","last_unstale":"2024-03-11T19:51:56.708988+0000","last_undegraded":"2024-03-11T19:51:56.708988+0000","last_fullsized":"2024-03-11T19:51:56.708988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:03:34.690444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.708926+0000","last_change":"2024-03-11T19:49:55.238517+0000","last_active":"2024-03-11T19:51:56.708926+0000","last_peered":"2024-03-11T19:51:56.708926+0000","last_clean":"2024-03-11T19:51:56.708926+0000","last_became_active":"2024-03-11T19:49:55.238242+0000","last_became_peered":"2024-03-11T19:49:55.238242+0000","last_unstale":"2024-03-11T19:51:56.708926+0000","last_undegraded":"2024-03-11T19:51:56.708926+0000","last_fullsized":"2024-03-11T19:51:56.708926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:19:27.755670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.539179+0000","last_change":"2024-03-11T19:49:55.222671+0000","last_active":"2024-03-11T19:51:56.539179+0000","last_peered":"2024-03-11T19:51:56.539179+0000","last_clean":"2024-03-11T19:51:56.539179+0000","last_became_active":"2024-03-11T19:49:55.221364+0000","last_became_peered":"2024-03-11T19:49:55.221364+0000","last_unstale":"2024-03-11T19:51:56.539179+0000","last_undegraded":"2024-03-11T19:51:56.539179+0000","last_fullsized":"2024-03-11T19:51:56.539179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:52:34.434573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.718416+0000","last_change":"2024-03-11T19:51:56.716335+0000","last_active":"2024-03-11T19:51:56.718416+0000","last_peered":"2024-03-11T19:51:56.718416+0000","last_clean":"2024-03-11T19:51:56.718416+0000","last_became_active":"2024-03-11T19:51:56.714786+0000","last_became_peered":"2024-03-11T19:51:56.714786+0000","last_unstale":"2024-03-11T19:51:56.718416+0000","last_undegraded":"2024-03-11T19:51:56.718416+0000","last_fullsized":"2024-03-11T19:51:56.718416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:39:36.271156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709769+0000","last_change":"2024-03-11T19:49:55.207412+0000","last_active":"2024-03-11T19:51:56.709769+0000","last_peered":"2024-03-11T19:51:56.709769+0000","last_clean":"2024-03-11T19:51:56.709769+0000","last_became_active":"2024-03-11T19:49:55.207108+0000","last_became_peered":"2024-03-11T19:49:55.207108+0000","last_unstale":"2024-03-11T19:51:56.709769+0000","last_undegraded":"2024-03-11T19:51:56.709769+0000","last_fullsized":"2024-03-11T19:51:56.709769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:24:28.162831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709567+0000","last_change":"2024-03-11T19:49:55.208948+0000","last_active":"2024-03-11T19:51:56.709567+0000","last_peered":"2024-03-11T19:51:56.709567+0000","last_clean":"2024-03-11T19:51:56.709567+0000","last_became_active":"2024-03-11T19:49:55.208635+0000","last_became_peered":"2024-03-11T19:49:55.208635+0000","last_unstale":"2024-03-11T19:51:56.709567+0000","last_undegraded":"2024-03-11T19:51:56.709567+0000","last_fullsized":"2024-03-11T19:51:56.709567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:46:04.244852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.527266+0000","last_change":"2024-03-11T19:50:55.488762+0000","last_active":"2024-03-11T19:51:56.527266+0000","last_peered":"2024-03-11T19:51:56.527266+0000","last_clean":"2024-03-11T19:51:56.527266+0000","last_became_active":"2024-03-11T19:50:55.488324+0000","last_became_peered":"2024-03-11T19:50:55.488324+0000","last_unstale":"2024-03-11T19:51:56.527266+0000","last_undegraded":"2024-03-11T19:51:56.527266+0000","last_fullsized":"2024-03-11T19:51:56.527266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:18:36.522393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.596155+0000","last_change":"2024-03-11T19:49:55.231203+0000","last_active":"2024-03-11T19:51:56.596155+0000","last_peered":"2024-03-11T19:51:56.596155+0000","last_clean":"2024-03-11T19:51:56.596155+0000","last_became_active":"2024-03-11T19:49:55.230977+0000","last_became_peered":"2024-03-11T19:49:55.230977+0000","last_unstale":"2024-03-11T19:51:56.596155+0000","last_undegraded":"2024-03-11T19:51:56.596155+0000","last_fullsized":"2024-03-11T19:51:56.596155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:35:26.296593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.526030+0000","last_change":"2024-03-11T19:49:55.233445+0000","last_active":"2024-03-11T19:51:56.526030+0000","last_peered":"2024-03-11T19:51:56.526030+0000","last_clean":"2024-03-11T19:51:56.526030+0000","last_became_active":"2024-03-11T19:49:55.229616+0000","last_became_peered":"2024-03-11T19:49:55.229616+0000","last_unstale":"2024-03-11T19:51:56.526030+0000","last_undegraded":"2024-03-11T19:51:56.526030+0000","last_fullsized":"2024-03-11T19:51:56.526030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:38:02.488420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709253+0000","last_change":"2024-03-11T19:49:55.238857+0000","last_active":"2024-03-11T19:51:56.709253+0000","last_peered":"2024-03-11T19:51:56.709253+0000","last_clean":"2024-03-11T19:51:56.709253+0000","last_became_active":"2024-03-11T19:49:55.238542+0000","last_became_peered":"2024-03-11T19:49:55.238542+0000","last_unstale":"2024-03-11T19:51:56.709253+0000","last_undegraded":"2024-03-11T19:51:56.709253+0000","last_fullsized":"2024-03-11T19:51:56.709253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:03:16.213894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.530815+0000","last_change":"2024-03-11T19:49:55.224053+0000","last_active":"2024-03-11T19:51:56.530815+0000","last_peered":"2024-03-11T19:51:56.530815+0000","last_clean":"2024-03-11T19:51:56.530815+0000","last_became_active":"2024-03-11T19:49:55.223666+0000","last_became_peered":"2024-03-11T19:49:55.223666+0000","last_unstale":"2024-03-11T19:51:56.530815+0000","last_undegraded":"2024-03-11T19:51:56.530815+0000","last_fullsized":"2024-03-11T19:51:56.530815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:52:43.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-11T19:51:56.709981+0000","last_change":"2024-03-11T19:49:55.217711+0000","last_active":"2024-03-11T19:51:56.709981+0000","last_peered":"2024-03-11T19:51:56.709981+0000","last_clean":"2024-03-11T19:51:56.709981+0000","last_became_active":"2024-03-11T19:49:55.217374+0000","last_became_peered":"2024-03-11T19:49:55.217374+0000","last_unstale":"2024-03-11T19:51:56.709981+0000","last_undegraded":"2024-03-11T19:51:56.709981+0000","last_fullsized":"2024-03-11T19:51:56.709981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:49:54.689954+0000","last_clean_scrub_stamp":"2024-03-11T19:49:54.689954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:13:46.059417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66698,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":679,"num_read_kb":412,"num_write":172,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1147,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":200094,"data_compressed":4842,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":537362,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1187840,"data_stored":1191260,"data_compressed":67061,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":537576,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070525,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":537474,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":1134078,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":532752,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1110016,"data_stored":594798,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":534140,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1110016,"data_stored":595302,"data_compressed":31245,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":532946,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":539088,"data_compressed":29631,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757326,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1129134,"data_compressed":65447,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34172,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35816,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62136,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62640,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64134,"data_compressed":1614,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:52:22.127 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-11T19:52:22.127 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-11T19:52:22.128 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-11T19:52:22.128 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-11T19:52:22.591 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:22.742 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:22 smithi107 ceph-mon[31712]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:22.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:22 smithi122 ceph-mon[36397]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:22.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:22 smithi145 ceph-mon[36730]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:52:23.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:23 smithi107 ceph-mon[31712]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:23 smithi122 ceph-mon[36397]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:23.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:23 smithi145 ceph-mon[36730]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 -- 172.21.15.107:0/3101077455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f942410e4b0 msgr2=0x7f94241108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 --2- 172.21.15.107:0/3101077455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f942410e4b0 0x7f94241108a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b0a0 tx=0x7f941c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 -- 172.21.15.107:0/3101077455 shutdown_connections 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 --2- 172.21.15.107:0/3101077455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f942410e4b0 0x7f94241108a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 --2- 172.21.15.107:0/3101077455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9424100900 0x7f9424100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 --2- 172.21.15.107:0/3101077455 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94241012b0 0x7f942410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 -- 172.21.15.107:0/3101077455 >> 172.21.15.107:0/3101077455 conn(0x7f94240fc540 msgr2=0x7f94240fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 -- 172.21.15.107:0/3101077455 shutdown_connections 2024-03-11T19:52:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.987+0000 7f942aa00700 1 -- 172.21.15.107:0/3101077455 wait complete. 2024-03-11T19:52:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.988+0000 7f942aa00700 1 Processor -- start 2024-03-11T19:52:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 -- start start 2024-03-11T19:52:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9424100900 0x7f942419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 0x7f942419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 0x7f942419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942419f290 con 0x7f94241012b0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942419f400 con 0x7f942410e4b0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942aa00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942419f5c0 con 0x7f9424100900 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f9428c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 0x7f942419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f9429600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9424100900 0x7f942419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f9428c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 0x7f942419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50080/0 (socket says 172.21.15.107:50080) 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f942a000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 0x7f942419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.989+0000 7f9428c00700 1 -- 172.21.15.107:0/2106628712 learned_addr learned my addr 172.21.15.107:0/2106628712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 -- 172.21.15.107:0/2106628712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9424100900 msgr2=0x7f942419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9424100900 0x7f942419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 -- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 msgr2=0x7f942419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 0x7f942419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 -- 172.21.15.107:0/2106628712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009d00 con 0x7f942410e4b0 2024-03-11T19:52:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942a000700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 0x7f942419ec80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b070 tx=0x7f941c0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00e040 con 0x7f942410e4b0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941c0094a0 con 0x7f942410e4b0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942419f840 con 0x7f942410e4b0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c005040 con 0x7f942410e4b0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.990+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94241a7100 con 0x7f942410e4b0 2024-03-11T19:52:24.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.991+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94241096d0 con 0x7f942410e4b0 2024-03-11T19:52:24.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.993+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f941c01d470 con 0x7f942410e4b0 2024-03-11T19:52:24.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.995+0000 7f9422000700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 0x7f940c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:25.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.995+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f941c0cef00 con 0x7f942410e4b0 2024-03-11T19:52:25.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.995+0000 7f9429600700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 0x7f940c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:25.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.996+0000 7f9429600700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 0x7f940c0a4000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9418009c60 tx=0x7f9418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:25.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:24.997+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f941c097d90 con 0x7f942410e4b0 2024-03-11T19:52:25.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.261+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f942419b710 con 0x7f942410e4b0 2024-03-11T19:52:25.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.262+0000 7f9422000700 1 -- 172.21.15.107:0/2106628712 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f941c09b450 con 0x7f942410e4b0 2024-03-11T19:52:25.267 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:25.267 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 msgr2=0x7f940c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 0x7f940c0a4000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9418009c60 tx=0x7f9418009260 comp rx=0 tx=0).stop 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 msgr2=0x7f942419ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 0x7f942419ec80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b070 tx=0x7f941c0077f0 comp rx=0 tx=0).stop 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 shutdown_connections 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f940c0a1b40 0x7f940c0a4000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f942410e4b0 0x7f942419ec80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:25.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9424100900 0x7f942419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:25.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 --2- 172.21.15.107:0/2106628712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94241012b0 0x7f942419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:25.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 >> 172.21.15.107:0/2106628712 conn(0x7f94240fc540 msgr2=0x7f942410f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:25.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 shutdown_connections 2024-03-11T19:52:25.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:25.267+0000 7f942aa00700 1 -- 172.21.15.107:0/2106628712 wait complete. 2024-03-11T19:52:25.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:25 smithi107 ceph-mon[31712]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:25.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:25 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2106628712' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:52:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:25 smithi122 ceph-mon[36397]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:25 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2106628712' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:52:25.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:25 smithi145 ceph-mon[36730]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:25.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:25 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2106628712' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:52:26.176 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-11T19:52:26.176 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:52:26.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.396+0000 7f80b7600640 1 -- 172.21.15.107:0/1304250169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b01033c0 msgr2=0x7f80b01037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 --2- 172.21.15.107:0/1304250169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b01033c0 0x7f80b01037e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8098009900 tx=0x7f809802bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 -- 172.21.15.107:0/1304250169 shutdown_connections 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 --2- 172.21.15.107:0/1304250169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b0104f70 0x7f80b010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 --2- 172.21.15.107:0/1304250169 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b01045b0 0x7f80b0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 --2- 172.21.15.107:0/1304250169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b01033c0 0x7f80b01037e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 -- 172.21.15.107:0/1304250169 >> 172.21.15.107:0/1304250169 conn(0x7f80b00fe970 msgr2=0x7f80b0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 -- 172.21.15.107:0/1304250169 shutdown_connections 2024-03-11T19:52:26.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.397+0000 7f80b7600640 1 -- 172.21.15.107:0/1304250169 wait complete. 2024-03-11T19:52:26.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 Processor -- start 2024-03-11T19:52:26.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 -- start start 2024-03-11T19:52:26.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b01033c0 0x7f80b0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01a0f60 con 0x7f80b0104f70 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01a10d0 con 0x7f80b01045b0 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.398+0000 7f80b7600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01a1290 con 0x7f80b01033c0 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80b5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80b5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50084/0 (socket says 172.21.15.107:50084) 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80b5600640 1 -- 172.21.15.107:0/3942976914 learned_addr learned my addr 172.21.15.107:0/3942976914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40412/0 (socket says 172.21.15.107:40412) 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 -- 172.21.15.107:0/3942976914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b01033c0 msgr2=0x7f80b0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b01033c0 0x7f80b0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 -- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 msgr2=0x7f80b01a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 -- 172.21.15.107:0/3942976914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098009560 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80b5600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80a7e00640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80a000e9f0 tx=0x7f80a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.399+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000ccc0 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.400+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80a000ce20 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.400+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0004a00 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.400+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b01a1570 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.400+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b01a1a30 con 0x7f80b01045b0 2024-03-11T19:52:26.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.401+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b010a760 con 0x7f80b01045b0 2024-03-11T19:52:26.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.403+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f80a00050b0 con 0x7f80b01045b0 2024-03-11T19:52:26.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.404+0000 7f80ae000640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 0x7f80900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.404+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f80a0014070 con 0x7f80b01045b0 2024-03-11T19:52:26.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.404+0000 7f80b4c00640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 0x7f80900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.405+0000 7f80b4c00640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 0x7f80900a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f809802c6a0 tx=0x7f8098000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.408+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80a0096640 con 0x7f80b01045b0 2024-03-11T19:52:26.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.607+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80b010edd0 con 0x7f80b01045b0 2024-03-11T19:52:26.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.609+0000 7f80ae000640 1 -- 172.21.15.107:0/3942976914 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7f80a0099d00 con 0x7f80b01045b0 2024-03-11T19:52:26.614 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:26.615 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 msgr2=0x7f80900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 0x7f80900a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f809802c6a0 tx=0x7f8098000bc0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 msgr2=0x7f80b01999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f80a000e9f0 tx=0x7f80a000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 shutdown_connections 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80900a1a30 0x7f80900a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80b01045b0 0x7f80b01999b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80b0104f70 0x7f80b01a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 --2- 172.21.15.107:0/3942976914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80b01033c0 0x7f80b0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 >> 172.21.15.107:0/3942976914 conn(0x7f80b00fe970 msgr2=0x7f80b0107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:26.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 shutdown_connections 2024-03-11T19:52:26.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.612+0000 7f80b7600640 1 -- 172.21.15.107:0/3942976914 wait complete. 2024-03-11T19:52:26.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:52:26.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 -- 172.21.15.107:0/410350287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c108990 msgr2=0x7f413c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 --2- 172.21.15.107:0/410350287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c108990 0x7f413c108e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f412c009ca0 tx=0x7f412c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 -- 172.21.15.107:0/410350287 shutdown_connections 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 --2- 172.21.15.107:0/410350287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c109350 0x7f413c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 --2- 172.21.15.107:0/410350287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c108990 0x7f413c108e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 --2- 172.21.15.107:0/410350287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c1077a0 0x7f413c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 -- 172.21.15.107:0/410350287 >> 172.21.15.107:0/410350287 conn(0x7f413c076740 msgr2=0x7f413c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:26.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 -- 172.21.15.107:0/410350287 shutdown_connections 2024-03-11T19:52:26.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.848+0000 7f4141800640 1 -- 172.21.15.107:0/410350287 wait complete. 2024-03-11T19:52:26.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 Processor -- start 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 -- start start 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c108990 0x7f413c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c109350 0x7f413c1a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c1a5320 con 0x7f413c109350 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c1a5490 con 0x7f413c1077a0 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.849+0000 7f4141800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c1a5650 con 0x7f413c108990 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40430/0 (socket says 172.21.15.107:40430) 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 -- 172.21.15.107:0/100738711 learned_addr learned my addr 172.21.15.107:0/100738711 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413a000640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c108990 0x7f413c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413b400640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c109350 0x7f413c1a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 -- 172.21.15.107:0/100738711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c108990 msgr2=0x7f413c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c108990 0x7f413c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 -- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c109350 msgr2=0x7f413c1a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c109350 0x7f413c1a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 -- 172.21.15.107:0/100738711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009900 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.850+0000 7f413aa00640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f412400e9c0 tx=0x7f412400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.851+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124009700 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.851+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f412400cd80 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.851+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c1a5930 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.851+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124010430 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.851+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c1a5e60 con 0x7f413c1077a0 2024-03-11T19:52:26.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.852+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c10eb40 con 0x7f413c1077a0 2024-03-11T19:52:26.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.854+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4124003680 con 0x7f413c1077a0 2024-03-11T19:52:26.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.856+0000 7f4133400640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 0x7f41200a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:26.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.856+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4124014070 con 0x7f413c1077a0 2024-03-11T19:52:26.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.856+0000 7f413a000640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 0x7f41200a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:26.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.857+0000 7f413a000640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 0x7f41200a3f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f412c006010 tx=0x7f412c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:26.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:26.859+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4124096920 con 0x7f413c1077a0 2024-03-11T19:52:27.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.051+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f413c113180 con 0x7f413c1077a0 2024-03-11T19:52:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.052+0000 7f4133400640 1 -- 172.21.15.107:0/100738711 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f413c113180 con 0x7f413c1077a0 2024-03-11T19:52:27.057 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:52:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.054+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 msgr2=0x7f41200a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.054+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 0x7f41200a3f70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f412c006010 tx=0x7f412c0326b0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.054+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 msgr2=0x7f413c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.054+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f412400e9c0 tx=0x7f412400ee80 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 shutdown_connections 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f41200a1ab0 0x7f41200a3f70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f413c109350 0x7f413c1a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f413c108990 0x7f413c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 --2- 172.21.15.107:0/100738711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413c1077a0 0x7f413c19d830 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 >> 172.21.15.107:0/100738711 conn(0x7f413c076740 msgr2=0x7f413c10bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 shutdown_connections 2024-03-11T19:52:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.055+0000 7f4141800640 1 -- 172.21.15.107:0/100738711 wait complete. 2024-03-11T19:52:27.078 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 -- 172.21.15.107:0/193611166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52201033c0 msgr2=0x7f52201037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 --2- 172.21.15.107:0/193611166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52201033c0 0x7f52201037e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5218009be0 tx=0x7f521802bef0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 -- 172.21.15.107:0/193611166 shutdown_connections 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 --2- 172.21.15.107:0/193611166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201045b0 0x7f5220104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 --2- 172.21.15.107:0/193611166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5220104f70 0x7f522010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 --2- 172.21.15.107:0/193611166 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f52201033c0 0x7f52201037e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 -- 172.21.15.107:0/193611166 >> 172.21.15.107:0/193611166 conn(0x7f52200fe930 msgr2=0x7f5220100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 -- 172.21.15.107:0/193611166 shutdown_connections 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.282+0000 7f5225a00640 1 -- 172.21.15.107:0/193611166 wait complete. 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.283+0000 7f5225a00640 1 Processor -- start 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.283+0000 7f5225a00640 1 -- start start 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.283+0000 7f5225a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 0x7f5220199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f5225a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f5225a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5220104f70 0x7f52201a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f5225a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52201a0f60 con 0x7f52201045b0 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f5225a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52201a10d0 con 0x7f52201033c0 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f5225a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52201a1290 con 0x7f5220104f70 2024-03-11T19:52:27.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50114/0 (socket says 172.21.15.107:50114) 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 -- 172.21.15.107:0/812033844 learned_addr learned my addr 172.21.15.107:0/812033844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521ea00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 0x7f5220199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521f400640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5220104f70 0x7f52201a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 -- 172.21.15.107:0/812033844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5220104f70 msgr2=0x7f52201a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5220104f70 0x7f52201a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 -- 172.21.15.107:0/812033844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 msgr2=0x7f5220199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 0x7f5220199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521e000640 1 -- 172.21.15.107:0/812033844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218009840 con 0x7f52201045b0 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.284+0000 7f521ea00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 0x7f5220199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f521e000640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e9c0 tx=0x7f520c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00ccc0 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f520c00ce20 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52201a1570 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c010430 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.285+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52201a1a80 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.286+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522010a6e0 con 0x7f52201045b0 2024-03-11T19:52:27.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.288+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f520c004f80 con 0x7f52201045b0 2024-03-11T19:52:27.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.290+0000 7f5217400640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 0x7f52040a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.290+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f520c014070 con 0x7f52201045b0 2024-03-11T19:52:27.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.290+0000 7f521ea00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 0x7f52040a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.291+0000 7f521ea00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 0x7f52040a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f521802c500 tx=0x7f52180333e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:27.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.294+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f520c096900 con 0x7f52201045b0 2024-03-11T19:52:27.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.493+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f522002cdd0 con 0x7f52201045b0 2024-03-11T19:52:27.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.494+0000 7f5217400640 1 -- 172.21.15.107:0/812033844 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f520c099fc0 con 0x7f52201045b0 2024-03-11T19:52:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:52:27.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 msgr2=0x7f52040a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 0x7f52040a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f521802c500 tx=0x7f52180333e0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 msgr2=0x7f52201999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f520c00e9c0 tx=0x7f520c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 shutdown_connections 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f52040a1a30 0x7f52040a3ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5220104f70 0x7f52201a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52201045b0 0x7f52201999b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 --2- 172.21.15.107:0/812033844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52201033c0 0x7f5220199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.497+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 >> 172.21.15.107:0/812033844 conn(0x7f52200fe930 msgr2=0x7f52201078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.498+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 shutdown_connections 2024-03-11T19:52:27.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.498+0000 7f5225a00640 1 -- 172.21.15.107:0/812033844 wait complete. 2024-03-11T19:52:27.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:52:27.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 -- 172.21.15.107:0/960235485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc1033a0 msgr2=0x7fa1cc1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/960235485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc1037c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009860 tx=0x7fa1c002bb70 comp rx=0 tx=0).stop 2024-03-11T19:52:27.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 -- 172.21.15.107:0/960235485 shutdown_connections 2024-03-11T19:52:27.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/960235485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc104f50 0x7fa1cc10b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/960235485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104590 0x7fa1cc104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/960235485 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc1037c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 -- 172.21.15.107:0/960235485 >> 172.21.15.107:0/960235485 conn(0x7fa1cc0fe930 msgr2=0x7fa1cc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:27.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 -- 172.21.15.107:0/960235485 shutdown_connections 2024-03-11T19:52:27.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.735+0000 7fa1d3e00640 1 -- 172.21.15.107:0/960235485 wait complete. 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.736+0000 7fa1d3e00640 1 Processor -- start 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 -- start start 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc104590 0x7fa1cc199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104f50 0x7fa1cc1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1a0f20 con 0x7fa1cc104f50 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1a1090 con 0x7fa1cc1033a0 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1a1250 con 0x7fa1cc104590 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104f50 0x7fa1cc1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40460/0 (socket says 172.21.15.107:40460) 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d1400640 1 -- 172.21.15.107:0/1631471309 learned_addr learned my addr 172.21.15.107:0/1631471309 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.737+0000 7fa1d0a00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc104590 0x7fa1cc199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 -- 172.21.15.107:0/1631471309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc104590 msgr2=0x7fa1cc199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc104590 0x7fa1cc199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 -- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104f50 msgr2=0x7fa1cc1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104f50 0x7fa1cc1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 -- 172.21.15.107:0/1631471309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c0009490 con 0x7fa1cc1033a0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d1400640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1c002c0b0 tx=0x7fa1c00045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c002c510 con 0x7fa1cc1033a0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1c002cb50 con 0x7fa1cc1033a0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc1a14d0 con 0x7fa1cc1033a0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0032600 con 0x7fa1cc1033a0 2024-03-11T19:52:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.738+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc1a1840 con 0x7fa1cc1033a0 2024-03-11T19:52:27.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.739+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc10a6c0 con 0x7fa1cc1033a0 2024-03-11T19:52:27.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.741+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa1c003b040 con 0x7fa1cc1033a0 2024-03-11T19:52:27.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.743+0000 7fa1ca000640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 0x7fa1b40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:27.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.743+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa1c00ef370 con 0x7fa1cc1033a0 2024-03-11T19:52:27.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.743+0000 7fa1d0a00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 0x7fa1b40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:27.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.743+0000 7fa1d0a00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 0x7fa1b40a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc19a940 tx=0x7fa1bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:27.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.745+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1c00b8170 con 0x7fa1cc1033a0 2024-03-11T19:52:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:27 smithi122 ceph-mon[36397]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3942976914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:27 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/100738711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:27.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:27 smithi145 ceph-mon[36730]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3942976914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:27.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:27 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/100738711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:27 smithi107 ceph-mon[31712]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3942976914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:27 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/100738711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:27.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.942+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1cc10edb0 con 0x7fa1cc1033a0 2024-03-11T19:52:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.943+0000 7fa1ca000640 1 -- 172.21.15.107:0/1631471309 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1c004b0a0 con 0x7fa1cc1033a0 2024-03-11T19:52:27.947 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:52:27.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 msgr2=0x7fa1b40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 0x7fa1b40a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc19a940 tx=0x7fa1bc009260 comp rx=0 tx=0).stop 2024-03-11T19:52:27.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 msgr2=0x7fa1cc199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:27.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1c002c0b0 tx=0x7fa1c00045a0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 shutdown_connections 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa1b40a1a30 0x7fa1b40a3ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1cc104f50 0x7fa1cc1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa1cc104590 0x7fa1cc199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 --2- 172.21.15.107:0/1631471309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1cc1033a0 0x7fa1cc199430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.946+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 >> 172.21.15.107:0/1631471309 conn(0x7fa1cc0fe930 msgr2=0x7fa1cc107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.947+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 shutdown_connections 2024-03-11T19:52:27.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:27.947+0000 7fa1d3e00640 1 -- 172.21.15.107:0/1631471309 wait complete. 2024-03-11T19:52:27.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:52:28.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 -- 172.21.15.107:0/1107588019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104590 msgr2=0x7f3088104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 --2- 172.21.15.107:0/1107588019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104590 0x7f3088104a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3074009900 tx=0x7f307402bc40 comp rx=0 tx=0).stop 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 -- 172.21.15.107:0/1107588019 shutdown_connections 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 --2- 172.21.15.107:0/1107588019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104f50 0x7f308810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 --2- 172.21.15.107:0/1107588019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104590 0x7f3088104a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 --2- 172.21.15.107:0/1107588019 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f30881037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 -- 172.21.15.107:0/1107588019 >> 172.21.15.107:0/1107588019 conn(0x7f30880fe930 msgr2=0x7f3088100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 -- 172.21.15.107:0/1107588019 shutdown_connections 2024-03-11T19:52:28.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.181+0000 7f308f400640 1 -- 172.21.15.107:0/1107588019 wait complete. 2024-03-11T19:52:28.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.182+0000 7f308f400640 1 Processor -- start 2024-03-11T19:52:28.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 -- start start 2024-03-11T19:52:28.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f3088199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104f50 0x7f30881a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a0f10 con 0x7f3088104f50 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a1080 con 0x7f3088104590 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308f400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881a1240 con 0x7f30881033a0 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104f50 0x7f30881a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f3087e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f3088199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f3087e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40478/0 (socket says 172.21.15.107:40478) 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f3087e00640 1 -- 172.21.15.107:0/858564752 learned_addr learned my addr 172.21.15.107:0/858564752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.183+0000 7f308ca00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f3088199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35046/0 (socket says 172.21.15.107:35046) 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 -- 172.21.15.107:0/858564752 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 msgr2=0x7f3088199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f3088199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 -- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104f50 msgr2=0x7f30881a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104f50 0x7f30881a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 -- 172.21.15.107:0/858564752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009490 con 0x7f3088104590 2024-03-11T19:52:28.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3087e00640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f3074005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:28.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740043d0 con 0x7f3088104590 2024-03-11T19:52:28.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30740052d0 con 0x7f3088104590 2024-03-11T19:52:28.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3074009560 con 0x7f3088104590 2024-03-11T19:52:28.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003cc0 con 0x7f3088104590 2024-03-11T19:52:28.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.184+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881a1840 con 0x7f3088104590 2024-03-11T19:52:28.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.185+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308810a6c0 con 0x7f3088104590 2024-03-11T19:52:28.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.187+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f3074005440 con 0x7f3088104590 2024-03-11T19:52:28.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.189+0000 7f3085600640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 0x7f30700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.189+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f30740efc90 con 0x7f3088104590 2024-03-11T19:52:28.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.189+0000 7f308ca00640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 0x7f30700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.190+0000 7f308ca00640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 0x7f30700a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30881043f0 tx=0x7f307c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:28.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.191+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30740b8a90 con 0x7f3088104590 2024-03-11T19:52:28.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.388+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f308810f000 con 0x7f3088104590 2024-03-11T19:52:28.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.390+0000 7f3085600640 1 -- 172.21.15.107:0/858564752 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f30740bc150 con 0x7f3088104590 2024-03-11T19:52:28.394 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 msgr2=0x7f30700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 0x7f30700a3ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30881043f0 tx=0x7f307c005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 msgr2=0x7f3088199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f3074005e80 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 shutdown_connections 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f30700a1a30 0x7f30700a3ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3088104590 0x7f3088199960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3088104f50 0x7f30881a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 --2- 172.21.15.107:0/858564752 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30881033a0 0x7f3088199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 >> 172.21.15.107:0/858564752 conn(0x7f30880fe930 msgr2=0x7f3088107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:28.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 shutdown_connections 2024-03-11T19:52:28.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.392+0000 7f308f400640 1 -- 172.21.15.107:0/858564752 wait complete. 2024-03-11T19:52:28.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:52:28.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 -- 172.21.15.107:0/3100830547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 msgr2=0x7f678c10b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 --2- 172.21.15.107:0/3100830547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c10b800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6784009be0 tx=0x7f678402bef0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 -- 172.21.15.107:0/3100830547 shutdown_connections 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 --2- 172.21.15.107:0/3100830547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c10b800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 --2- 172.21.15.107:0/3100830547 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1045b0 0x7f678c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 --2- 172.21.15.107:0/3100830547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1033c0 0x7f678c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 -- 172.21.15.107:0/3100830547 >> 172.21.15.107:0/3100830547 conn(0x7f678c0fe970 msgr2=0x7f678c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 -- 172.21.15.107:0/3100830547 shutdown_connections 2024-03-11T19:52:28.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.621+0000 7f6793600640 1 -- 172.21.15.107:0/3100830547 wait complete. 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.622+0000 7f6793600640 1 Processor -- start 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.622+0000 7f6793600640 1 -- start start 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.622+0000 7f6793600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.622+0000 7f6793600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1045b0 0x7f678c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6793600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6793600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a0f10 con 0x7f678c104f70 2024-03-11T19:52:28.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6793600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a1080 con 0x7f678c1045b0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6793600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a1240 con 0x7f678c1033c0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6791600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35064/0 (socket says 172.21.15.107:35064) 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 -- 172.21.15.107:0/3206823474 learned_addr learned my addr 172.21.15.107:0/3206823474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f678be00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1045b0 0x7f678c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 -- 172.21.15.107:0/3206823474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1045b0 msgr2=0x7f678c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1045b0 0x7f678c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 -- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 msgr2=0x7f678c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c1a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6790c00640 1 -- 172.21.15.107:0/3206823474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780009490 con 0x7f678c1033c0 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.623+0000 7f6791600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6790c00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9c0 tx=0x7f678000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000cde0 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6780010910 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780018a10 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6778009560 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.624+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6784009840 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.625+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c10a680 con 0x7f678c1033c0 2024-03-11T19:52:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.627+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6780010a80 con 0x7f678c1033c0 2024-03-11T19:52:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.628+0000 7f6789600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 0x7f67740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.628+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6780014070 con 0x7f678c1033c0 2024-03-11T19:52:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.628+0000 7f678be00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 0x7f67740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.629+0000 7f678be00640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 0x7f67740a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f678c19a930 tx=0x7f6778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.633+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6780096ce0 con 0x7f678c1033c0 2024-03-11T19:52:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:28 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/812033844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:28 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1631471309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:28 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/858564752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:28 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/812033844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:28 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1631471309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:28 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/858564752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:28 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/812033844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:28 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1631471309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:28 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/858564752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:28.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.869+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f678c02ce40 con 0x7f678c1033c0 2024-03-11T19:52:28.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.871+0000 7f6789600640 1 -- 172.21.15.107:0/3206823474 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f678009a3a0 con 0x7f678c1033c0 2024-03-11T19:52:28.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:28.876 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:52:28.876 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 msgr2=0x7f67740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 0x7f67740a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f678c19a930 tx=0x7f6778009260 comp rx=0 tx=0).stop 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 msgr2=0x7f678c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9c0 tx=0x7f678000ee80 comp rx=0 tx=0).stop 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 shutdown_connections 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f67740a1a30 0x7f67740a3ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f678c1045b0 0x7f678c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c104f70 0x7f678c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 --2- 172.21.15.107:0/3206823474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f678c1033c0 0x7f678c199420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:28.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 >> 172.21.15.107:0/3206823474 conn(0x7f678c0fe970 msgr2=0x7f678c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:28.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 shutdown_connections 2024-03-11T19:52:28.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:28.873+0000 7f6793600640 1 -- 172.21.15.107:0/3206823474 wait complete. 2024-03-11T19:52:28.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.085+0000 7fcd0c000640 1 -- 172.21.15.107:0/4183059923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04077720 msgr2=0x7fcd04077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.085+0000 7fcd0c000640 1 --2- 172.21.15.107:0/4183059923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04077720 0x7fcd04077fc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009ca0 tx=0x7fccfc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 -- 172.21.15.107:0/4183059923 shutdown_connections 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 --2- 172.21.15.107:0/4183059923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04076d80 0x7fcd040771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 --2- 172.21.15.107:0/4183059923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04077720 0x7fcd04077fc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 --2- 172.21.15.107:0/4183059923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04075b40 0x7fcd04075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 -- 172.21.15.107:0/4183059923 >> 172.21.15.107:0/4183059923 conn(0x7fcd040fe930 msgr2=0x7fcd04100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 -- 172.21.15.107:0/4183059923 shutdown_connections 2024-03-11T19:52:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.086+0000 7fcd0c000640 1 -- 172.21.15.107:0/4183059923 wait complete. 2024-03-11T19:52:29.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 Processor -- start 2024-03-11T19:52:29.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 -- start start 2024-03-11T19:52:29.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 0x7fcd0419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04077720 0x7fcd041a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041a52c0 con 0x7fcd04075b40 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041a5430 con 0x7fcd04077720 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0c000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041a55f0 con 0x7fcd04076d80 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd08c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd09600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 0x7fcd0419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd08c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35078/0 (socket says 172.21.15.107:35078) 2024-03-11T19:52:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd08c00640 1 -- 172.21.15.107:0/1219528774 learned_addr learned my addr 172.21.15.107:0/1219528774 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.087+0000 7fcd0a000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04077720 0x7fcd041a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 -- 172.21.15.107:0/1219528774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 msgr2=0x7fcd0419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 -- 172.21.15.107:0/1219528774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04077720 msgr2=0x7fcd041a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04077720 0x7fcd041a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 -- 172.21.15.107:0/1219528774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf0009490 con 0x7fcd04075b40 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd08c00640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd09600640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 0x7fcd0419d7d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e9f0 tx=0x7fccf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800ccc0 con 0x7fcd04075b40 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf800ce20 con 0x7fcd04075b40 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8004a00 con 0x7fcd04075b40 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc009900 con 0x7fcd04075b40 2024-03-11T19:52:29.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.088+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041a5c00 con 0x7fcd04075b40 2024-03-11T19:52:29.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.090+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0410fe80 con 0x7fcd04075b40 2024-03-11T19:52:29.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.091+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fccf80050b0 con 0x7fcd04075b40 2024-03-11T19:52:29.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.093+0000 7fcd02000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 0x7fccec0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.093+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fccf8014070 con 0x7fcd04075b40 2024-03-11T19:52:29.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.093+0000 7fcd08c00640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 0x7fccec0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.093+0000 7fcd08c00640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 0x7fccec0a3f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd0419ece0 tx=0x7fccf0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.097+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fccf8096530 con 0x7fcd04075b40 2024-03-11T19:52:29.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.293+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd041131c0 con 0x7fcd04075b40 2024-03-11T19:52:29.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.297+0000 7fcd02000640 1 -- 172.21.15.107:0/1219528774 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7fccf8099bf0 con 0x7fcd04075b40 2024-03-11T19:52:29.303 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 msgr2=0x7fccec0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 0x7fccec0a3f70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd0419ece0 tx=0x7fccf0005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 msgr2=0x7fcd0419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 0x7fcd0419d7d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e9f0 tx=0x7fccf800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 shutdown_connections 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fccec0a1ab0 0x7fccec0a3f70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd04077720 0x7fcd041a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd04076d80 0x7fcd0419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 --2- 172.21.15.107:0/1219528774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd04075b40 0x7fcd0419d7d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.300+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 >> 172.21.15.107:0/1219528774 conn(0x7fcd040fe930 msgr2=0x7fcd040fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.301+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 shutdown_connections 2024-03-11T19:52:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.301+0000 7fcd0c000640 1 -- 172.21.15.107:0/1219528774 wait complete. 2024-03-11T19:52:29.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:52:29.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 -- 172.21.15.107:0/1210200668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c102370 msgr2=0x7f185c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 --2- 172.21.15.107:0/1210200668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c102370 0x7f185c1047e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 -- 172.21.15.107:0/1210200668 shutdown_connections 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 --2- 172.21.15.107:0/1210200668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c104d20 0x7f185c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 --2- 172.21.15.107:0/1210200668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c102370 0x7f185c1047e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 --2- 172.21.15.107:0/1210200668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c0ffa00 0x7f185c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 -- 172.21.15.107:0/1210200668 >> 172.21.15.107:0/1210200668 conn(0x7f185c0fb620 msgr2=0x7f185c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 -- 172.21.15.107:0/1210200668 shutdown_connections 2024-03-11T19:52:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.528+0000 7f1864400640 1 -- 172.21.15.107:0/1210200668 wait complete. 2024-03-11T19:52:29.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 Processor -- start 2024-03-11T19:52:29.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 -- start start 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c0ffa00 0x7f185c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 0x7f185c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 0x7f185c1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1a5310 con 0x7f185c104d20 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1a5480 con 0x7f185c0ffa00 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.529+0000 7f1864400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1a5640 con 0x7f185c102370 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 0x7f185c1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1861000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 0x7f185c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1861000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 0x7f185c19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35094/0 (socket says 172.21.15.107:35094) 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1861000640 1 -- 172.21.15.107:0/2332832814 learned_addr learned my addr 172.21.15.107:0/2332832814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1861a00640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c0ffa00 0x7f185c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 -- 172.21.15.107:0/2332832814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 msgr2=0x7f185c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 0x7f185c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 -- 172.21.15.107:0/2332832814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c0ffa00 msgr2=0x7f185c19d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c0ffa00 0x7f185c19d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 -- 172.21.15.107:0/2332832814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848009560 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f1862400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 0x7f185c1a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fd0 tx=0x7f185400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.530+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185400ce20 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.531+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c1a5920 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.531+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1854005020 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.531+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854010690 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.531+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c1a5de0 con 0x7f185c104d20 2024-03-11T19:52:29.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.532+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c0fd110 con 0x7f185c104d20 2024-03-11T19:52:29.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.534+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f1854005500 con 0x7f185c104d20 2024-03-11T19:52:29.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.535+0000 7f185a000640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 0x7f18440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.535+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1854014070 con 0x7f185c104d20 2024-03-11T19:52:29.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.536+0000 7f1861a00640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 0x7f18440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.536+0000 7f1861a00640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 0x7f18440a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1850009bd0 tx=0x7f1850005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.539+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18540d2070 con 0x7f185c104d20 2024-03-11T19:52:29.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.736+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f185c02ce40 con 0x7f185c104d20 2024-03-11T19:52:29.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.737+0000 7f185a000640 1 -- 172.21.15.107:0/2332832814 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1854096d70 con 0x7f185c104d20 2024-03-11T19:52:29.742 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 msgr2=0x7f18440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 0x7f18440a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1850009bd0 tx=0x7f1850005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 msgr2=0x7f185c1a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 0x7f185c1a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fd0 tx=0x7f185400ef10 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 shutdown_connections 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f18440a1a30 0x7f18440a3ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f185c104d20 0x7f185c1a4dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f185c102370 0x7f185c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 --2- 172.21.15.107:0/2332832814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f185c0ffa00 0x7f185c19d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 >> 172.21.15.107:0/2332832814 conn(0x7f185c0fb620 msgr2=0x7f185c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 shutdown_connections 2024-03-11T19:52:29.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.742+0000 7f1864400640 1 -- 172.21.15.107:0/2332832814 wait complete. 2024-03-11T19:52:29.762 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:29 smithi107 ceph-mon[31712]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:29.763 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:29 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3206823474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.763 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:29 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1219528774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:29 smithi122 ceph-mon[36397]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:29 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3206823474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:29 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1219528774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:52:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:29 smithi145 ceph-mon[36730]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:29 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3206823474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:29 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1219528774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:29.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.979+0000 7fea7cc00640 1 -- 172.21.15.107:0/3981239941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea781077a0 msgr2=0x7fea78107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.979+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3981239941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea781077a0 0x7fea78107bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fea68009ca0 tx=0x7fea6802c100 comp rx=0 tx=0).stop 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 -- 172.21.15.107:0/3981239941 shutdown_connections 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3981239941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78109350 0x7fea7810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3981239941 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea78108990 0x7fea78108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3981239941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea781077a0 0x7fea78107bc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 -- 172.21.15.107:0/3981239941 >> 172.21.15.107:0/3981239941 conn(0x7fea78076740 msgr2=0x7fea78078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 -- 172.21.15.107:0/3981239941 shutdown_connections 2024-03-11T19:52:29.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.980+0000 7fea7cc00640 1 -- 172.21.15.107:0/3981239941 wait complete. 2024-03-11T19:52:29.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 Processor -- start 2024-03-11T19:52:29.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 -- start start 2024-03-11T19:52:29.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 0x7fea7819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78108990 0x7fea7819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 0x7fea781a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781a5360 con 0x7fea78109350 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781a54d0 con 0x7fea78108990 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.981+0000 7fea7cc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781a5690 con 0x7fea781077a0 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 0x7fea781a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea75e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 0x7fea7819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea75e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 0x7fea7819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35120/0 (socket says 172.21.15.107:35120) 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea75e00640 1 -- 172.21.15.107:0/3972950018 learned_addr learned my addr 172.21.15.107:0/3972950018 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea6d400640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78108990 0x7fea7819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 -- 172.21.15.107:0/3972950018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 msgr2=0x7fea7819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 0x7fea7819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 -- 172.21.15.107:0/3972950018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78108990 msgr2=0x7fea7819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78108990 0x7fea7819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:29.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 -- 172.21.15.107:0/3972950018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68009900 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.982+0000 7fea76800640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 0x7fea781a4e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e970 tx=0x7fea7000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.983+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70009620 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.983+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7000ccc0 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.983+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70004a00 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.983+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea781a5970 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.983+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea781a5e30 con 0x7fea78109350 2024-03-11T19:52:29.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.984+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7810eac0 con 0x7fea78109350 2024-03-11T19:52:29.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.986+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fea70005020 con 0x7fea78109350 2024-03-11T19:52:29.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.987+0000 7fea6f400640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 0x7fea5c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:29.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.987+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fea70014070 con 0x7fea78109350 2024-03-11T19:52:29.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.988+0000 7fea75e00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 0x7fea5c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:29.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.988+0000 7fea75e00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 0x7fea5c0a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea68005e30 tx=0x7fea680326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:29.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:29.991+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea700965b0 con 0x7fea78109350 2024-03-11T19:52:30.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.187+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea7802ce40 con 0x7fea78109350 2024-03-11T19:52:30.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.188+0000 7fea6f400640 1 -- 172.21.15.107:0/3972950018 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea70099c70 con 0x7fea78109350 2024-03-11T19:52:30.193 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:52:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 msgr2=0x7fea5c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 0x7fea5c0a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea68005e30 tx=0x7fea680326d0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 msgr2=0x7fea781a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 0x7fea781a4e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e970 tx=0x7fea7000ee30 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 shutdown_connections 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fea5c0a1a30 0x7fea5c0a3ef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea78108990 0x7fea7819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea78109350 0x7fea781a4e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 --2- 172.21.15.107:0/3972950018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea781077a0 0x7fea7819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 >> 172.21.15.107:0/3972950018 conn(0x7fea78076740 msgr2=0x7fea7810bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 shutdown_connections 2024-03-11T19:52:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.191+0000 7fea7cc00640 1 -- 172.21.15.107:0/3972950018 wait complete. 2024-03-11T19:52:30.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 -- 172.21.15.107:0/258439208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac0ff9f0 msgr2=0x7f2cac0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 --2- 172.21.15.107:0/258439208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac0ffe10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 -- 172.21.15.107:0/258439208 shutdown_connections 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 --2- 172.21.15.107:0/258439208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac100350 0x7f2cac1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 --2- 172.21.15.107:0/258439208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100cf0 0x7f2cac101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 --2- 172.21.15.107:0/258439208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac0ffe10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 -- 172.21.15.107:0/258439208 >> 172.21.15.107:0/258439208 conn(0x7f2cac0fb610 msgr2=0x7f2cac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 -- 172.21.15.107:0/258439208 shutdown_connections 2024-03-11T19:52:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.423+0000 7f2cb2400640 1 -- 172.21.15.107:0/258439208 wait complete. 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.424+0000 7f2cb2400640 1 Processor -- start 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.424+0000 7f2cb2400640 1 -- start start 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100350 0x7f2cac199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1a0f20 con 0x7f2cac100cf0 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1a1090 con 0x7f2cac0ff9f0 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cb2400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1a1250 con 0x7f2cac100350 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cabe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50232/0 (socket says 172.21.15.107:50232) 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cabe00640 1 -- 172.21.15.107:0/3258717468 learned_addr learned my addr 172.21.15.107:0/3258717468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cab400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2caaa00640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100350 0x7f2cac199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.425+0000 7f2cabe00640 1 -- 172.21.15.107:0/3258717468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100350 msgr2=0x7f2cac199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cabe00640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100350 0x7f2cac199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cabe00640 1 -- 172.21.15.107:0/3258717468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac0ff9f0 msgr2=0x7f2cac199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cabe00640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cabe00640 1 -- 172.21.15.107:0/3258717468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c009560 con 0x7f2cac100cf0 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cabe00640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e970 tx=0x7f2ca400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4009720 con 0x7f2cac100cf0 2024-03-11T19:52:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4005500 con 0x7f2cac100cf0 2024-03-11T19:52:30.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac1a1530 con 0x7f2cac100cf0 2024-03-11T19:52:30.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4010430 con 0x7f2cac100cf0 2024-03-11T19:52:30.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.426+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1a1a40 con 0x7f2cac100cf0 2024-03-11T19:52:30.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.427+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac10a880 con 0x7f2cac100cf0 2024-03-11T19:52:30.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.429+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f2ca4004ff0 con 0x7f2cac100cf0 2024-03-11T19:52:30.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.431+0000 7f2ca3e00640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 0x7f2c8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.431+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ca4014070 con 0x7f2cac100cf0 2024-03-11T19:52:30.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.431+0000 7f2cab400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 0x7f2c8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.432+0000 7f2cab400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 0x7f2c8c0a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c000c00 tx=0x7f2c9c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:30.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.435+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ca4096950 con 0x7f2cac100cf0 2024-03-11T19:52:30.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.634+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cac02ce40 con 0x7f2cac100cf0 2024-03-11T19:52:30.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.635+0000 7f2ca3e00640 1 -- 172.21.15.107:0/3258717468 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ca409a010 con 0x7f2cac100cf0 2024-03-11T19:52:30.640 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:52:30.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 msgr2=0x7f2c8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 0x7f2c8c0a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c000c00 tx=0x7f2c9c02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 msgr2=0x7f2cac1a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e970 tx=0x7f2ca400ed70 comp rx=0 tx=0).stop 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 shutdown_connections 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f2c8c0a1a30 0x7f2c8c0a3ef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cac100cf0 0x7f2cac1a09e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2cac100350 0x7f2cac199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 --2- 172.21.15.107:0/3258717468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cac0ff9f0 0x7f2cac199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.637+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 >> 172.21.15.107:0/3258717468 conn(0x7f2cac0fb610 msgr2=0x7f2cac0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.638+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 shutdown_connections 2024-03-11T19:52:30.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.638+0000 7f2cb2400640 1 -- 172.21.15.107:0/3258717468 wait complete. 2024-03-11T19:52:30.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:52:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:30 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2332832814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:30 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3972950018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:30 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2332832814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:30 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3972950018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 -- 172.21.15.107:0/3902339059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341033c0 msgr2=0x7fa4341037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3902339059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341033c0 0x7fa4341037e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa428009900 tx=0x7fa42802bc40 comp rx=0 tx=0).stop 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 -- 172.21.15.107:0/3902339059 shutdown_connections 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3902339059 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa434104f70 0x7fa43410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3902339059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4341045b0 0x7fa434104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3902339059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341033c0 0x7fa4341037e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 -- 172.21.15.107:0/3902339059 >> 172.21.15.107:0/3902339059 conn(0x7fa4340fe930 msgr2=0x7fa434100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 -- 172.21.15.107:0/3902339059 shutdown_connections 2024-03-11T19:52:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.872+0000 7fa43bc00640 1 -- 172.21.15.107:0/3902339059 wait complete. 2024-03-11T19:52:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.873+0000 7fa43bc00640 1 Processor -- start 2024-03-11T19:52:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.873+0000 7fa43bc00640 1 -- start start 2024-03-11T19:52:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 0x7fa434079470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 0x7fa434075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434076230 con 0x7fa434104f70 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340763a0 con 0x7fa4341045b0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa43bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434076560 con 0x7fa4341033c0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa439200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa439c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 0x7fa434075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa439200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35152/0 (socket says 172.21.15.107:35152) 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa439200640 1 -- 172.21.15.107:0/3652269514 learned_addr learned my addr 172.21.15.107:0/3652269514 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.874+0000 7fa433e00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 0x7fa434079470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 -- 172.21.15.107:0/3652269514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 msgr2=0x7fa434079470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 0x7fa434079470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 -- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 msgr2=0x7fa434075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 0x7fa434075aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 -- 172.21.15.107:0/3652269514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009490 con 0x7fa4341033c0 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa433e00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 0x7fa434079470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439c00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 0x7fa434075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:30.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa439200640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa428000c00 tx=0x7fa428005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:30.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280043d0 con 0x7fa4341033c0 2024-03-11T19:52:30.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4280052d0 con 0x7fa4341033c0 2024-03-11T19:52:30.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c009840 con 0x7fa4341033c0 2024-03-11T19:52:30.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.875+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428003ca0 con 0x7fa4341033c0 2024-03-11T19:52:30.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.876+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa428009560 con 0x7fa4341033c0 2024-03-11T19:52:30.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.877+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43410a760 con 0x7fa4341033c0 2024-03-11T19:52:30.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.879+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa428005440 con 0x7fa4341033c0 2024-03-11T19:52:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.880+0000 7fa431600640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 0x7fa41c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.880+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa4280f0440 con 0x7fa4341033c0 2024-03-11T19:52:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.880+0000 7fa433e00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 0x7fa41c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.881+0000 7fa433e00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 0x7fa41c0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa434076eb0 tx=0x7fa420005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:30.882+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4280b9240 con 0x7fa4341033c0 2024-03-11T19:52:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:30 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2332832814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:30 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3972950018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.079+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa43402cda0 con 0x7fa4341033c0 2024-03-11T19:52:31.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.080+0000 7fa431600640 1 -- 172.21.15.107:0/3652269514 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa428043020 con 0x7fa4341033c0 2024-03-11T19:52:31.085 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 msgr2=0x7fa41c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 0x7fa41c0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa434076eb0 tx=0x7fa420005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 msgr2=0x7fa434078f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa428000c00 tx=0x7fa428005e80 comp rx=0 tx=0).stop 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 shutdown_connections 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa41c0a1a30 0x7fa41c0a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4341045b0 0x7fa434079470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa434104f70 0x7fa434075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 --2- 172.21.15.107:0/3652269514 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa4341033c0 0x7fa434078f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 >> 172.21.15.107:0/3652269514 conn(0x7fa4340fe930 msgr2=0x7fa434107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 shutdown_connections 2024-03-11T19:52:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.083+0000 7fa43bc00640 1 -- 172.21.15.107:0/3652269514 wait complete. 2024-03-11T19:52:31.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 -- 172.21.15.107:0/2596801728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8984109370 msgr2=0x7f898410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 --2- 172.21.15.107:0/2596801728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8984109370 0x7f898410fc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f897c009ca0 tx=0x7f897c02c100 comp rx=0 tx=0).stop 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 -- 172.21.15.107:0/2596801728 shutdown_connections 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 --2- 172.21.15.107:0/2596801728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8984109370 0x7f898410fc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 --2- 172.21.15.107:0/2596801728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 0x7f8984108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 --2- 172.21.15.107:0/2596801728 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89841077c0 0x7f8984107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.321+0000 7f898b400640 1 -- 172.21.15.107:0/2596801728 >> 172.21.15.107:0/2596801728 conn(0x7f8984076790 msgr2=0x7f8984078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.322+0000 7f898b400640 1 -- 172.21.15.107:0/2596801728 shutdown_connections 2024-03-11T19:52:31.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.322+0000 7f898b400640 1 -- 172.21.15.107:0/2596801728 wait complete. 2024-03-11T19:52:31.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 Processor -- start 2024-03-11T19:52:31.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 -- start start 2024-03-11T19:52:31.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 0x7f898419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89841a52c0 con 0x7f89841089b0 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89841a5430 con 0x7f89841077c0 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.323+0000 7f898b400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89841a55f0 con 0x7f8984109370 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f897be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 0x7f898419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8989400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8989400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35166/0 (socket says 172.21.15.107:35166) 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40600/0 (socket says 172.21.15.107:40600) 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8989400640 1 -- 172.21.15.107:0/2303867236 learned_addr learned my addr 172.21.15.107:0/2303867236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 -- 172.21.15.107:0/2303867236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 msgr2=0x7f89841a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 -- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 msgr2=0x7f898419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 0x7f898419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 -- 172.21.15.107:0/2303867236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c009900 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8989400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.324+0000 7f8988a00640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f897400e9c0 tx=0x7f897400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.325+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400ccc0 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.325+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f897400ce20 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.325+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89841a58d0 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.325+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974010430 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.325+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89841a5d90 con 0x7f89841077c0 2024-03-11T19:52:31.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.326+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898410eae0 con 0x7f89841077c0 2024-03-11T19:52:31.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.328+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f8974004000 con 0x7f89841077c0 2024-03-11T19:52:31.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.329+0000 7f8982000640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 0x7f89640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.330+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8974014070 con 0x7f89841077c0 2024-03-11T19:52:31.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.330+0000 7f897be00640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 0x7f89640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.330+0000 7f897be00640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 0x7f89640a3ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f898419ece0 tx=0x7f896c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:31.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.333+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f89740969b0 con 0x7f89841077c0 2024-03-11T19:52:31.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.571+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89841131c0 con 0x7f89841077c0 2024-03-11T19:52:31.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.572+0000 7f8982000640 1 -- 172.21.15.107:0/2303867236 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f897409a070 con 0x7f89841077c0 2024-03-11T19:52:31.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:31.577 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:52:31.578 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:52:31.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.575+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 msgr2=0x7f89640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.575+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 0x7f89640a3ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f898419ece0 tx=0x7f896c005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.575+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 msgr2=0x7f898419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.575+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f897400e9c0 tx=0x7f897400ee80 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 shutdown_connections 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89640a1a30 0x7f89640a3ef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8984109370 0x7f89841a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89841089b0 0x7f898419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 --2- 172.21.15.107:0/2303867236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89841077c0 0x7f898419d7d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 >> 172.21.15.107:0/2303867236 conn(0x7f8984076790 msgr2=0x7f898410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 shutdown_connections 2024-03-11T19:52:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.576+0000 7f898b400640 1 -- 172.21.15.107:0/2303867236 wait complete. 2024-03-11T19:52:31.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:52:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:31 smithi145 ceph-mon[36730]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:31 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3258717468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:31 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3652269514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:31 smithi107 ceph-mon[31712]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:31 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3258717468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:31 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3652269514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:31.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 -- 172.21.15.107:0/748007821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80181077c0 msgr2=0x7f8018107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 --2- 172.21.15.107:0/748007821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80181077c0 0x7f8018107be0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f800c009900 tx=0x7f800c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:31.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 -- 172.21.15.107:0/748007821 shutdown_connections 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 --2- 172.21.15.107:0/748007821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 0x7f8018108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 --2- 172.21.15.107:0/748007821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8018109370 0x7f801810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 --2- 172.21.15.107:0/748007821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80181077c0 0x7f8018107be0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 -- 172.21.15.107:0/748007821 >> 172.21.15.107:0/748007821 conn(0x7f8018076790 msgr2=0x7f8018078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 -- 172.21.15.107:0/748007821 shutdown_connections 2024-03-11T19:52:31.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.807+0000 7f801f600640 1 -- 172.21.15.107:0/748007821 wait complete. 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.808+0000 7f801f600640 1 Processor -- start 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 -- start start 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80181077c0 0x7f801819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 0x7f801819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181a5370 con 0x7f8018109370 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181a54e0 con 0x7f80181089b0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801f600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181a56a0 con 0x7f80181077c0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50316/0 (socket says 172.21.15.107:50316) 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801d600640 1 -- 172.21.15.107:0/2222766951 learned_addr learned my addr 172.21.15.107:0/2222766951 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f8017e00640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 0x7f801819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.809+0000 7f801cc00640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80181077c0 0x7f801819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 -- 172.21.15.107:0/2222766951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80181077c0 msgr2=0x7f801819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80181077c0 0x7f801819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 -- 172.21.15.107:0/2222766951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 msgr2=0x7f801819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 0x7f801819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 -- 172.21.15.107:0/2222766951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c009560 con 0x7f8018109370 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801d600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f801000eb10 tx=0x7f801000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:31.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010004b20 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8010004c80 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80181a5980 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010010650 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.810+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80181a5ee0 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.812+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801810ea80 con 0x7f8018109370 2024-03-11T19:52:31.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.813+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f8010005020 con 0x7f8018109370 2024-03-11T19:52:31.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.815+0000 7f8015600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 0x7f80000a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:31.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.815+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8010014070 con 0x7f8018109370 2024-03-11T19:52:31.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.816+0000 7f801cc00640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 0x7f80000a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:31.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.816+0000 7f801cc00640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 0x7f80000a3f70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f800c02c240 tx=0x7f800c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:31.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:31.820+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8010096af0 con 0x7f8018109370 2024-03-11T19:52:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:31 smithi122 ceph-mon[36397]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:31 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3258717468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:52:32.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:31 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3652269514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:52:32.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.019+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8018113210 con 0x7f8018109370 2024-03-11T19:52:32.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.021+0000 7f8015600640 1 -- 172.21.15.107:0/2222766951 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7f801009a1b0 con 0x7f8018109370 2024-03-11T19:52:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:32.027 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 msgr2=0x7f80000a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 0x7f80000a3f70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f800c02c240 tx=0x7f800c02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 msgr2=0x7f80181a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f801000eb10 tx=0x7f801000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 shutdown_connections 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f80000a1ab0 0x7f80000a3f70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181089b0 0x7f801819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8018109370 0x7f80181a4e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 --2- 172.21.15.107:0/2222766951 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80181077c0 0x7f801819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 >> 172.21.15.107:0/2222766951 conn(0x7f8018076790 msgr2=0x7f801810bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 shutdown_connections 2024-03-11T19:52:32.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.024+0000 7f801f600640 1 -- 172.21.15.107:0/2222766951 wait complete. 2024-03-11T19:52:32.048 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 -- 172.21.15.107:0/3956303678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc102380 msgr2=0x7fe0dc102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/3956303678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc102380 0x7fe0dc102800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009900 tx=0x7fe0c402bc50 comp rx=0 tx=0).stop 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 -- 172.21.15.107:0/3956303678 shutdown_connections 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/3956303678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102d40 0x7fe0dc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/3956303678 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc102380 0x7fe0dc102800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/3956303678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc101190 0x7fe0dc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 -- 172.21.15.107:0/3956303678 >> 172.21.15.107:0/3956303678 conn(0x7fe0dc0fc760 msgr2=0x7fe0dc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 -- 172.21.15.107:0/3956303678 shutdown_connections 2024-03-11T19:52:32.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.256+0000 7fe0e2e00640 1 -- 172.21.15.107:0/3956303678 wait complete. 2024-03-11T19:52:32.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 Processor -- start 2024-03-11T19:52:32.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 -- start start 2024-03-11T19:52:32.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc101190 0x7fe0dc074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc102d40 0x7fe0dc073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc074210 con 0x7fe0dc102380 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc074380 con 0x7fe0dc102d40 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.257+0000 7fe0e2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1a12b0 con 0x7fe0dc101190 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50346/0 (socket says 172.21.15.107:50346) 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 -- 172.21.15.107:0/2682896686 learned_addr learned my addr 172.21.15.107:0/2682896686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0dbe00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc101190 0x7fe0dc074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0e0e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc102d40 0x7fe0dc073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 -- 172.21.15.107:0/2682896686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc101190 msgr2=0x7fe0dc074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc101190 0x7fe0dc074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 -- 172.21.15.107:0/2682896686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc102d40 msgr2=0x7fe0dc073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc102d40 0x7fe0dc073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 -- 172.21.15.107:0/2682896686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4009560 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.258+0000 7fe0db400640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4036040 tx=0x7fe0c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.259+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40043d0 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.259+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc1a1500 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.259+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0c40052d0 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.259+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4003cc0 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.259+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc1a1a10 con 0x7fe0dc102380 2024-03-11T19:52:32.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.260+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc1084b0 con 0x7fe0dc102380 2024-03-11T19:52:32.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.262+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe0c4005440 con 0x7fe0dc102380 2024-03-11T19:52:32.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.265+0000 7fe0d8c00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 0x7fe0c00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.266+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe0c40efa60 con 0x7fe0dc102380 2024-03-11T19:52:32.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.266+0000 7fe0dbe00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 0x7fe0c00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.266+0000 7fe0dbe00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 0x7fe0c00a3ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc1021e0 tx=0x7fe0cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.269+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0c40b88a0 con 0x7fe0dc102380 2024-03-11T19:52:32.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.525+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0dc10cba0 con 0x7fe0dc102380 2024-03-11T19:52:32.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.527+0000 7fe0d8c00640 1 -- 172.21.15.107:0/2682896686 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fe0c40b8070 con 0x7fe0dc102380 2024-03-11T19:52:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:32.532 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:52:32.533 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.530+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 msgr2=0x7fe0c00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.530+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 0x7fe0c00a3ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc1021e0 tx=0x7fe0cc005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.530+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 msgr2=0x7fe0dc073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.530+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4036040 tx=0x7fe0c4005e80 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 shutdown_connections 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe0c00a1a30 0x7fe0c00a3ef0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0dc102d40 0x7fe0dc073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc102380 0x7fe0dc073570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 --2- 172.21.15.107:0/2682896686 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe0dc101190 0x7fe0dc074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 >> 172.21.15.107:0/2682896686 conn(0x7fe0dc0fc760 msgr2=0x7fe0dc105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 shutdown_connections 2024-03-11T19:52:32.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.531+0000 7fe0e2e00640 1 -- 172.21.15.107:0/2682896686 wait complete. 2024-03-11T19:52:32.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:49:55.178113+0000', 'modified': '2024-03-11T19:50:29.435340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24590, 'mds_1': 24583, 'mds_2': 24547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/3941680931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 3941680931}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 3941680931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/2305866182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 2305866182}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 2305866182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6837/547863696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 547863696}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 547863696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.145:6839/322644420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 322644420}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 322644420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24583': {'gid': 24583, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.107:6841/981878935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 981878935}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 981878935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24590': {'gid': 24590, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6839/1167549353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1167549353}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1167549353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:52:32.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:52:32.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 -- 172.21.15.107:0/2082689774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 msgr2=0x7fdfdc108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/2082689774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc108e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc402bc40 comp rx=0 tx=0).stop 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 -- 172.21.15.107:0/2082689774 shutdown_connections 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/2082689774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc109350 0x7fdfdc10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/2082689774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc108e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/2082689774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 -- 172.21.15.107:0/2082689774 >> 172.21.15.107:0/2082689774 conn(0x7fdfdc076740 msgr2=0x7fdfdc078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 -- 172.21.15.107:0/2082689774 shutdown_connections 2024-03-11T19:52:32.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.754+0000 7fdfe1a00640 1 -- 172.21.15.107:0/2082689774 wait complete. 2024-03-11T19:52:32.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 Processor -- start 2024-03-11T19:52:32.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 -- start start 2024-03-11T19:52:32.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc109350 0x7fdfdc1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1a5310 con 0x7fdfdc108990 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1a5480 con 0x7fdfdc1077a0 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.755+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc1a5640 con 0x7fdfdc109350 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc109350 0x7fdfdc1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40638/0 (socket says 172.21.15.107:40638) 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 -- 172.21.15.107:0/915544622 learned_addr learned my addr 172.21.15.107:0/915544622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50372/0 (socket says 172.21.15.107:50372) 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 -- 172.21.15.107:0/915544622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc109350 msgr2=0x7fdfdc1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc109350 0x7fdfdc1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 -- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 msgr2=0x7fdfdc19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 -- 172.21.15.107:0/915544622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4009560 con 0x7fdfdc1077a0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.756+0000 7fdfdaa00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.757+0000 7fdfd3400640 1 -- 172.21.15.107:0/915544622 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc009c30 con 0x7fdfdc1077a0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.757+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc1a5920 con 0x7fdfdc1077a0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.757+0000 7fdfd3400640 1 -- 172.21.15.107:0/915544622 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc009d90 con 0x7fdfdc1077a0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.757+0000 7fdfd3400640 1 -- 172.21.15.107:0/915544622 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00f790 con 0x7fdfdc1077a0 2024-03-11T19:52:32.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.757+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc1a5e00 con 0x7fdfdc1077a0 2024-03-11T19:52:32.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.760+0000 7fdfd3400640 1 -- 172.21.15.107:0/915544622 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fdfcc010ad0 con 0x7fdfdc1077a0 2024-03-11T19:52:32.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.760+0000 7fdfe47e8740 1 Processor -- start 2024-03-11T19:52:32.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.760+0000 7fdfe47e8740 1 -- start start 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.760+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075f9f70 0x55e2075fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075ff0e0 0x55e207601570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207568b30 con 0x55e2075f9f70 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207566060 con 0x55e2075f9580 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207561840 con 0x55e2075ff0e0 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075f9f70 0x55e2075fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40644/0 (socket says 172.21.15.107:40644) 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 -- 172.21.15.107:0/226941041 learned_addr learned my addr 172.21.15.107:0/226941041 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075ff0e0 0x55e207601570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 -- 172.21.15.107:0/226941041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075ff0e0 msgr2=0x55e207601570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075ff0e0 0x55e207601570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 -- 172.21.15.107:0/226941041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075f9f70 msgr2=0x55e2075fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075f9f70 0x55e2075fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.761+0000 7fdfdb400640 1 -- 172.21.15.107:0/226941041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2075294c0 con 0x55e2075f9580 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfdb400640 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009f50 tx=0x7fdfd40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce279427d85dd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfd3400640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 0x7fdfc00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfcbe00640 1 -- 172.21.15.107:0/226941041 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4004b20 con 0x55e2075f9580 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfcbe00640 1 -- 172.21.15.107:0/226941041 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfd4004c80 con 0x55e2075f9580 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfcbe00640 1 -- 172.21.15.107:0/226941041 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005020 con 0x55e2075f9580 2024-03-11T19:52:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfd3400640 1 -- 172.21.15.107:0/915544622 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdfcc0cd9e0 con 0x7fdfdc1077a0 2024-03-11T19:52:32.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.762+0000 7fdfdaa00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 0x7fdfc00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- 172.21.15.107:0/226941041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 msgr2=0x55e2075f99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfdaa00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 0x7fdfc00a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0cb0d0 tx=0x7fdfcc0d3530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009f50 tx=0x7fdfd40118d0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- 172.21.15.107:0/226941041 shutdown_connections 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075ff0e0 0x55e207601570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075f9f70 0x55e2075fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/226941041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2075f99a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- 172.21.15.107:0/226941041 >> 172.21.15.107:0/226941041 conn(0x55e2075f4af0 msgr2=0x55e2075f4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- 172.21.15.107:0/226941041 shutdown_connections 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- 172.21.15.107:0/226941041 wait complete. 2024-03-11T19:52:32.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.763+0000 7fdfe47e8740 1 -- start start 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 Processor -- start 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2076033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e20760a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207506ea0 con 0x55e2075ff0e0 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207506b40 con 0x55e2075f9580 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2072a9f20 con 0x55e2075f9f70 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfe47e8740 10 client.0 _ll_register_callbacks cb 0x55e20751a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e20760a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35260/0 (socket says 172.21.15.107:35260) 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfd3e00640 1 -- 172.21.15.107:0/3540075099 learned_addr learned my addr 172.21.15.107:0/3540075099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.764+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2076033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 -- 172.21.15.107:0/3540075099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 msgr2=0x55e2076033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2076033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 -- 172.21.15.107:0/3540075099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 msgr2=0x55e20760a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e20760a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 -- 172.21.15.107:0/3540075099 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4009c00 con 0x55e2075f9f70 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402c850 tx=0x7fdfc4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11cd2d7b23956a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:52:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4004df0 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e207585c40 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.765+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdfc4004f50 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc4005110 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 10 client.25189 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e20755fec0 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e207585c40 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdfc403c040 con 0x55e2075f9f70 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 10 client.25189 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 10 client.25189 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:52:32.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 20 client.25189 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20760f360 0x55e20760ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 4 client.25189 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.766+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 --> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2072616c0 con 0x55e20760f360 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.767+0000 7fdfdb400640 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20760f360 0x55e20760ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.767+0000 7fdfdb400640 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20760f360 0x55e20760ec90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2ddd593986ceb63 server_cookie=d6da84106cc20b25 in_seq=0 out_seq=0 2024-03-11T19:52:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.767+0000 7fdfc8c00640 10 client.25189 ms_handle_connect on v2:172.21.15.145:6838/322644420 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.768+0000 7fdfc8c00640 1 -- 172.21.15.107:0/3540075099 <== mds.2 v2:172.21.15.145:6838/322644420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2072616c0 con 0x55e20760f360 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.768+0000 7fdfc8c00640 10 client.25189 handle_command_reply: tid=0 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.768+0000 7fdfe47e8740 1 client.25189 shutdown 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 msgr2=0x55e207603910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402c850 tx=0x7fdfc4004270 comp rx=0 tx=0).stop 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 shutdown_connections 2024-03-11T19:52:32.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e20760a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9f70 0x55e207603910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20760f360 0x55e20760ec90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/3540075099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9580 0x55e2076033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 >> 172.21.15.107:0/3540075099 conn(0x55e2075f4af0 msgr2=0x55e2075f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfc8c00640 0 client.25189 ms_handle_reset on v2:172.21.15.145:6838/322644420 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 shutdown_connections 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 1 -- 172.21.15.107:0/3540075099 wait complete. 2024-03-11T19:52:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.769+0000 7fdfe47e8740 20 client.25189 trim_cache size 0 max 16384 2024-03-11T19:52:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:32 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2303867236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:32.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:32 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2222766951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 Processor -- start 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 -- start start 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e2075f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2075f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207603fe0 con 0x55e2075ff0e0 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2075f55e0 con 0x55e2075f9f70 2024-03-11T19:52:32.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.786+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20760d840 con 0x55e2075f9580 2024-03-11T19:52:32.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e2075f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e2075f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35264/0 (socket says 172.21.15.107:35264) 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfdb400640 1 -- 172.21.15.107:0/993964106 learned_addr learned my addr 172.21.15.107:0/993964106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40664/0 (socket says 172.21.15.107:40664) 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfdb400640 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2075f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 -- 172.21.15.107:0/993964106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 msgr2=0x55e2075f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e2075f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 -- 172.21.15.107:0/993964106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 msgr2=0x55e2075f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2075f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 -- 172.21.15.107:0/993964106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2074fc8b0 con 0x55e2075f9f70 2024-03-11T19:52:32.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402c180 tx=0x7fdfc4004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc25aa0fa9100da server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.787+0000 7fdfcb400640 1 -- 172.21.15.107:0/993964106 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40035f0 con 0x55e2075f9f70 2024-03-11T19:52:32.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfcb400640 1 -- 172.21.15.107:0/993964106 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4034790 con 0x55e2075f9f70 2024-03-11T19:52:32.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfcb400640 1 -- 172.21.15.107:0/993964106 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4005050 con 0x55e2075f9f70 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 -- 172.21.15.107:0/993964106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 msgr2=0x55e20760a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402c180 tx=0x7fdfc4004a60 comp rx=0 tx=0).stop 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 -- 172.21.15.107:0/993964106 shutdown_connections 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e20760a910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2075f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/993964106 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e2075f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 -- 172.21.15.107:0/993964106 >> 172.21.15.107:0/993964106 conn(0x55e2075f4af0 msgr2=0x55e2075f4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.788+0000 7fdfe47e8740 1 -- 172.21.15.107:0/993964106 shutdown_connections 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.789+0000 7fdfe47e8740 1 -- 172.21.15.107:0/993964106 wait complete. 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.789+0000 7fdfe47e8740 1 -- start start 2024-03-11T19:52:32.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.789+0000 7fdfe47e8740 1 Processor -- start 2024-03-11T19:52:32.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:32 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2303867236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:32.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:32 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2222766951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:32.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e207607570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2076655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20760aad0 con 0x55e2075ff0e0 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20760ac40 con 0x55e2075f9f70 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e207665ae0 con 0x55e2075f9580 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfe47e8740 10 client.0 _ll_register_callbacks cb 0x7fdfc402cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e207607570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40676/0 (socket says 172.21.15.107:40676) 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfdb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e207607570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35276/0 (socket says 172.21.15.107:35276) 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 -- 172.21.15.107:0/1736933817 learned_addr learned my addr 172.21.15.107:0/1736933817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfdb400640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2076655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 -- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 msgr2=0x55e207607570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e207607570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 -- 172.21.15.107:0/1736933817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 msgr2=0x55e2076655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:32.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2076655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfd3e00640 1 -- 172.21.15.107:0/1736933817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e20755fec0 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.790+0000 7fdfdb400640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2076655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfd3e00640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005cc0 tx=0x7fdfc4003db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0fcc0a97fa9303b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4005d50 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdfc4000e40 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e207610370 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc4034440 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfe47e8740 10 client.25190 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.791+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e207522510 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4069020 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdfc403c040 con 0x55e2075f9f70 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 10 client.25190 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 10 client.25190 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:52:32.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 20 client.25190 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20766a7e0 0x55e207666cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 4 client.25190 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 --> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fdfd4004c80 con 0x55e20766a7e0 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.792+0000 7fdfdb400640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20766a7e0 0x55e207666cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfc8c00640 10 client.25190 ms_handle_connect on v2:172.21.15.145:6838/322644420 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfdb400640 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20766a7e0 0x55e207666cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b8084ed372d599c server_cookie=8daf19d10ed5c64a in_seq=0 out_seq=0 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfc8c00640 1 -- 172.21.15.107:0/1736933817 <== mds.2 v2:172.21.15.145:6838/322644420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fdfd4004c80 con 0x55e20766a7e0 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfc8c00640 10 client.25190 handle_command_reply: tid=0 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 client.25190 shutdown 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 msgr2=0x55e207607ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4005cc0 tx=0x7fdfc4003db0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 shutdown_connections 2024-03-11T19:52:32.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x55e20766a7e0 0x55e207666cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2075f9f70 0x55e207607ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfc8c00640 0 client.25190 ms_handle_reset on v2:172.21.15.145:6838/322644420 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2075ff0e0 0x55e2076655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 --2- 172.21.15.107:0/1736933817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55e2075f9580 0x55e207607570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.793+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 >> 172.21.15.107:0/1736933817 conn(0x55e2075f4af0 msgr2=0x55e2075f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 shutdown_connections 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe47e8740 1 -- 172.21.15.107:0/1736933817 wait complete. 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe47e8740 20 client.25190 trim_cache size 0 max 16384 2024-03-11T19:52:32.803 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24547, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 124.36736109100001, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 174.60046500199999 2024-03-11T19:52:32.804 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:52:32.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 msgr2=0x7fdfc00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 0x7fdfc00a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0cb0d0 tx=0x7fdfcc0d3530 comp rx=0 tx=0).stop 2024-03-11T19:52:32.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 msgr2=0x7fdfdc19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:32.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00de10 comp rx=0 tx=0).stop 2024-03-11T19:52:32.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 shutdown_connections 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fdfc00a1a30 0x7fdfc00a3ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdfdc109350 0x7fdfdc1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfdc108990 0x7fdfdc19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 --2- 172.21.15.107:0/915544622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfdc1077a0 0x7fdfdc19d820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 >> 172.21.15.107:0/915544622 conn(0x7fdfdc076740 msgr2=0x7fdfdc10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.794+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 shutdown_connections 2024-03-11T19:52:32.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:32.795+0000 7fdfe1a00640 1 -- 172.21.15.107:0/915544622 wait complete. 2024-03-11T19:52:32.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-11T19:52:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:32 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2303867236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:32 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2222766951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1232220565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 msgr2=0x7fa2c010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1232220565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c010b7e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009ca0 tx=0x7fa2b802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1232220565 shutdown_connections 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1232220565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c010b7e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1232220565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c0104590 0x7fa2c0104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1232220565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c01033a0 0x7fa2c01037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1232220565 >> 172.21.15.107:0/1232220565 conn(0x7fa2c00fe930 msgr2=0x7fa2c0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1232220565 shutdown_connections 2024-03-11T19:52:33.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.028+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1232220565 wait complete. 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 Processor -- start 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 -- start start 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c01033a0 0x7fa2c01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a0ec0 con 0x7fa2c0104f50 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a1030 con 0x7fa2c0104590 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.029+0000 7fa2c5e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a11f0 con 0x7fa2c01033a0 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2bf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:40702/0 (socket says 172.21.15.107:40702) 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 -- 172.21.15.107:0/1643587178 learned_addr learned my addr 172.21.15.107:0/1643587178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2bea00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c01033a0 0x7fa2c01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 -- 172.21.15.107:0/1643587178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c01033a0 msgr2=0x7fa2c01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c01033a0 0x7fa2c01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 -- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 msgr2=0x7fa2c01a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c01a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 -- 172.21.15.107:0/1643587178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0009490 con 0x7fa2c0104590 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2bf400640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.030+0000 7fa2be000640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9f0 tx=0x7fa2ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.031+0000 7fa2c4a00640 1 -- 172.21.15.107:0/1643587178 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac009720 con 0x7fa2c0104590 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.031+0000 7fa2c4a00640 1 -- 172.21.15.107:0/1643587178 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac00cd80 con 0x7fa2c0104590 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.031+0000 7fa2c4a00640 1 -- 172.21.15.107:0/1643587178 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac010690 con 0x7fa2c0104590 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.031+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b8009900 con 0x7fa2c0104590 2024-03-11T19:52:33.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.031+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c01a1800 con 0x7fa2c0104590 2024-03-11T19:52:33.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c8bba740 1 Processor -- start 2024-03-11T19:52:33.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c4a00640 1 -- 172.21.15.107:0/1643587178 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fa2ac0046c0 con 0x7fa2c0104590 2024-03-11T19:52:33.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c8bba740 1 -- start start 2024-03-11T19:52:33.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4ef70 0x5628a4e53a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.034+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e540e0 0x5628a4e56570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4dbdb30 con 0x5628a4e540e0 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4dbb060 con 0x5628a4e4ef70 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4db6840 con 0x5628a4e4e580 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35292/0 (socket says 172.21.15.107:35292) 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 -- 172.21.15.107:0/30684448 learned_addr learned my addr 172.21.15.107:0/30684448 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bea00640 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4ef70 0x5628a4e53a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2be000640 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e540e0 0x5628a4e56570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 -- 172.21.15.107:0/30684448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4ef70 msgr2=0x5628a4e53a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4ef70 0x5628a4e53a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 -- 172.21.15.107:0/30684448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e540e0 msgr2=0x5628a4e56570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e540e0 0x5628a4e56570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.035+0000 7fa2bf400640 1 -- 172.21.15.107:0/30684448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628a4d7e4c0 con 0x5628a4e4e580 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2c4a00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 0x7fa2a40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2bf400640 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa2b802c700 tx=0x7fa2b80093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a38f7813b6636 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2c4a00640 1 -- 172.21.15.107:0/1643587178 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa2ac014070 con 0x7fa2c0104590 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2bea00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 0x7fa2a40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2abe00640 1 -- 172.21.15.107:0/30684448 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8032d70 con 0x5628a4e4e580 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2abe00640 1 -- 172.21.15.107:0/30684448 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8034a90 con 0x5628a4e4e580 2024-03-11T19:52:33.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.036+0000 7fa2abe00640 1 -- 172.21.15.107:0/30684448 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003ea0 con 0x5628a4e4e580 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- 172.21.15.107:0/30684448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 msgr2=0x5628a4e4e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2bea00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 0x7fa2a40a3ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5628a4e4f370 tx=0x7fa2b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa2b802c700 tx=0x7fa2b80093f0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- 172.21.15.107:0/30684448 shutdown_connections 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4ef70 0x5628a4e53a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e540e0 0x5628a4e56570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/30684448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4e580 0x5628a4e4e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- 172.21.15.107:0/30684448 >> 172.21.15.107:0/30684448 conn(0x5628a4e49af0 msgr2=0x5628a4e49ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- 172.21.15.107:0/30684448 shutdown_connections 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- 172.21.15.107:0/30684448 wait complete. 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.037+0000 7fa2c8bba740 1 -- start start 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 Processor -- start 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e540e0 0x5628a4e5f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4d5bea0 con 0x5628a4e4ef70 2024-03-11T19:52:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4d5bb40 con 0x5628a4e4e580 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4afef20 con 0x5628a4e540e0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2c8bba740 10 client.0 _ll_register_callbacks cb 0x5628a4d6f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2be000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2be000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50442/0 (socket says 172.21.15.107:50442) 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2be000640 1 -- 172.21.15.107:0/3519601649 learned_addr learned my addr 172.21.15.107:0/3519601649 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2bf400640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e540e0 0x5628a4e5f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.038+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 -- 172.21.15.107:0/3519601649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e540e0 msgr2=0x5628a4e5f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e540e0 0x5628a4e5f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 -- 172.21.15.107:0/3519601649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 msgr2=0x5628a4e583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 -- 172.21.15.107:0/3519601649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c010c0e0 con 0x5628a4e4ef70 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2be000640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0a2950 tx=0x7fa2ac09a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c324d0f23f93949c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0ba2c0 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa2ac0cfe40 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.039+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628a4ddac40 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2ac09a6e0 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 10 client.15087 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5628a4db4ec0 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5628a4ddac40 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa2ac01f060 con 0x5628a4e4ef70 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 10 client.15087 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 10 client.15087 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 20 client.15087 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e64350 0x5628a4e68e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 4 client.15087 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 --> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628a4ab66c0 con 0x5628a4e64350 2024-03-11T19:52:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.040+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e64350 0x5628a4e68e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.041+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e64350 0x5628a4e68e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=256eb2a7776c049d server_cookie=19491616b56e833d in_seq=0 out_seq=0 2024-03-11T19:52:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.041+0000 7fa2a8c00640 10 client.15087 ms_handle_connect on v2:172.21.15.107:6840/981878935 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3519601649 <== mds.1 v2:172.21.15.107:6840/981878935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628a4ab66c0 con 0x5628a4e64350 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2a8c00640 10 client.15087 handle_command_reply: tid=0 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 client.15087 shutdown 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 msgr2=0x5628a4e588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0a2950 tx=0x7fa2ac09a2e0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 shutdown_connections 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e64350 0x5628a4e68e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e540e0 0x5628a4e5f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4ef70 0x5628a4e588e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2a8c00640 0 client.15087 ms_handle_reset on v2:172.21.15.107:6840/981878935 2024-03-11T19:52:33.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3519601649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 >> 172.21.15.107:0/3519601649 conn(0x5628a4e49af0 msgr2=0x5628a4e48e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 shutdown_connections 2024-03-11T19:52:33.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3519601649 wait complete. 2024-03-11T19:52:33.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.042+0000 7fa2c8bba740 20 client.15087 trim_cache size 0 max 16384 2024-03-11T19:52:33.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.059+0000 7fa2c8bba740 1 Processor -- start 2024-03-11T19:52:33.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.059+0000 7fa2c8bba740 1 -- start start 2024-03-11T19:52:33.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 0x5628a4e6d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e4aa90 con 0x5628a4e4e580 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e45100 con 0x5628a4e49af0 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e56700 con 0x5628a4e4ef70 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50448/0 (socket says 172.21.15.107:50448) 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 -- 172.21.15.107:0/3654110916 learned_addr learned my addr 172.21.15.107:0/3654110916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2bf400640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 0x5628a4e6d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 -- 172.21.15.107:0/3654110916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 msgr2=0x5628a4e48580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e48580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 -- 172.21.15.107:0/3654110916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 msgr2=0x5628a4e6d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 0x5628a4e6d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2be000640 1 -- 172.21.15.107:0/3654110916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628a4d518b0 con 0x5628a4e4e580 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.060+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 0x5628a4e6d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.061+0000 7fa2be000640 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac096b40 tx=0x7fa2ac0969e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33a17ece90cab13 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.061+0000 7fa2ab400640 1 -- 172.21.15.107:0/3654110916 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0a2be0 con 0x5628a4e4e580 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.061+0000 7fa2ab400640 1 -- 172.21.15.107:0/3654110916 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac0d3140 con 0x5628a4e4e580 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.061+0000 7fa2ab400640 1 -- 172.21.15.107:0/3654110916 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0850b0 con 0x5628a4e4e580 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.061+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3654110916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 msgr2=0x5628a4e5f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac096b40 tx=0x7fa2ac0969e0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3654110916 shutdown_connections 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e4e580 0x5628a4e5f8e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3654110916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e49af0 0x5628a4e6d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3654110916 >> 172.21.15.107:0/3654110916 conn(0x5628a4e4a240 msgr2=0x5628a4e6ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3654110916 shutdown_connections 2024-03-11T19:52:33.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3654110916 wait complete. 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 -- start start 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.062+0000 7fa2c8bba740 1 Processor -- start 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e5d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e5e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e601e0 con 0x5628a4e49af0 2024-03-11T19:52:33.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e64bc0 con 0x5628a4e4e580 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628a4e5e7a0 con 0x5628a4e4ef70 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2c8bba740 10 client.0 _ll_register_callbacks cb 0x7fa2ac004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50454/0 (socket says 172.21.15.107:50454) 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 -- 172.21.15.107:0/3718004460 learned_addr learned my addr 172.21.15.107:0/3718004460 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bf400640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e5e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2be000640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e5d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 -- 172.21.15.107:0/3718004460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 msgr2=0x5628a4e5e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e5e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 -- 172.21.15.107:0/3718004460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 msgr2=0x5628a4e5d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e5d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.063+0000 7fa2bea00640 1 -- 172.21.15.107:0/3718004460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b001d750 con 0x5628a4e49af0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2bea00640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5628a4e4f9e0 tx=0x7fa2b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd241c1066d00198 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0023070 con 0x5628a4e49af0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa2b002a510 con 0x5628a4e49af0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628a4e64d00 con 0x5628a4e49af0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b0017070 con 0x5628a4e49af0 2024-03-11T19:52:33.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2c8bba740 10 client.15093 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5628a4db4ec0 con 0x5628a4e49af0 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.064+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5628a4e64d00 con 0x5628a4e49af0 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa2b0029040 con 0x5628a4e49af0 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 10 client.15093 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 10 client.15093 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 20 client.15093 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e540e0 0x5628a4ebcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 4 client.15093 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:52:33.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 --> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa2b8034a90 con 0x5628a4e540e0 2024-03-11T19:52:33.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.065+0000 7fa2be000640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e540e0 0x5628a4ebcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2be000640 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e540e0 0x5628a4ebcca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3733a966e49f68b2 server_cookie=77b31d1c41adc2c4 in_seq=0 out_seq=0 2024-03-11T19:52:33.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2a8c00640 10 client.15093 ms_handle_connect on v2:172.21.15.107:6840/981878935 2024-03-11T19:52:33.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2a8c00640 1 -- 172.21.15.107:0/3718004460 <== mds.1 v2:172.21.15.107:6840/981878935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fa2b8034a90 con 0x5628a4e540e0 2024-03-11T19:52:33.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2a8c00640 10 client.15093 handle_command_reply: tid=0 2024-03-11T19:52:33.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 client.15093 shutdown 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 msgr2=0x5628a4e5cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5628a4e4f9e0 tx=0x7fa2b0017000 comp rx=0 tx=0).stop 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 shutdown_connections 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5628a4e540e0 0x5628a4ebcca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628a4e4e580 0x5628a4e5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5628a4e4ef70 0x5628a4e5e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 --2- 172.21.15.107:0/3718004460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628a4e49af0 0x5628a4e5cd50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2a8c00640 0 client.15093 ms_handle_reset on v2:172.21.15.107:6840/981878935 2024-03-11T19:52:33.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 >> 172.21.15.107:0/3718004460 conn(0x5628a4e4a240 msgr2=0x5628a4e6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 shutdown_connections 2024-03-11T19:52:33.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 1 -- 172.21.15.107:0/3718004460 wait complete. 2024-03-11T19:52:33.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.066+0000 7fa2c8bba740 20 client.15093 trim_cache size 0 max 16384 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24583, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 125.652876646, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 166.60778198099999 2024-03-11T19:52:33.084 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:52:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 msgr2=0x7fa2a40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 0x7fa2a40a3ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5628a4e4f370 tx=0x7fa2b0005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 msgr2=0x7fa2c0199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00e9f0 tx=0x7fa2ac00edf0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 shutdown_connections 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fa2a40a1a30 0x7fa2a40a3ef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c0104590 0x7fa2c0199910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c0104f50 0x7fa2c01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 --2- 172.21.15.107:0/1643587178 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa2c01033a0 0x7fa2c01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 >> 172.21.15.107:0/1643587178 conn(0x7fa2c00fe930 msgr2=0x7fa2c0107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 shutdown_connections 2024-03-11T19:52:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.067+0000 7fa2c5e00640 1 -- 172.21.15.107:0/1643587178 wait complete. 2024-03-11T19:52:33.089 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 -- 172.21.15.107:0/2837772243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 msgr2=0x7f07b8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 --2- 172.21.15.107:0/2837772243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8104a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009900 tx=0x7f07a002bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 -- 172.21.15.107:0/2837772243 shutdown_connections 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 --2- 172.21.15.107:0/2837772243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8104a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 --2- 172.21.15.107:0/2837772243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b8104f70 0x7f07b810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 --2- 172.21.15.107:0/2837772243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b81033c0 0x7f07b81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 -- 172.21.15.107:0/2837772243 >> 172.21.15.107:0/2837772243 conn(0x7f07b80fe930 msgr2=0x7f07b8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 -- 172.21.15.107:0/2837772243 shutdown_connections 2024-03-11T19:52:33.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.295+0000 7f07bd800640 1 -- 172.21.15.107:0/2837772243 wait complete. 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.296+0000 7f07bd800640 1 Processor -- start 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 -- start start 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b81033c0 0x7f07b8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b81a0f40 con 0x7f07b8104f70 2024-03-11T19:52:33.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b81a10b0 con 0x7f07b81045b0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07bd800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b81a1270 con 0x7f07b81033c0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b81033c0 0x7f07b8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50458/0 (socket says 172.21.15.107:50458) 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07b7400640 1 -- 172.21.15.107:0/1024191818 learned_addr learned my addr 172.21.15.107:0/1024191818 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.297+0000 7f07ae000640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 -- 172.21.15.107:0/1024191818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b81033c0 msgr2=0x7f07b8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b81033c0 0x7f07b8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 -- 172.21.15.107:0/1024191818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 msgr2=0x7f07b8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 -- 172.21.15.107:0/1024191818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0009560 con 0x7f07b8104f70 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07ae000640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07b7400640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e970 tx=0x7f07b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07afe00640 1 -- 172.21.15.107:0/1024191818 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0009720 con 0x7f07b8104f70 2024-03-11T19:52:33.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07afe00640 1 -- 172.21.15.107:0/1024191818 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07b0005500 con 0x7f07b8104f70 2024-03-11T19:52:33.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b81a1550 con 0x7f07b8104f70 2024-03-11T19:52:33.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07afe00640 1 -- 172.21.15.107:0/1024191818 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0010430 con 0x7f07b8104f70 2024-03-11T19:52:33.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.298+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b81a1a30 con 0x7f07b8104f70 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07afe00640 1 -- 172.21.15.107:0/1024191818 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f07b0004ff0 con 0x7f07b8104f70 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 Processor -- start 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 -- start start 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bacc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366ba33b30 con 0x56366bac4f70 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366ba31060 con 0x56366bac4580 2024-03-11T19:52:33.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.302+0000 7f07c05a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366ba2c840 con 0x56366baca0e0 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07ae000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50460/0 (socket says 172.21.15.107:50460) 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 -- 172.21.15.107:0/948513878 learned_addr learned my addr 172.21.15.107:0/948513878 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07ae000640 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bacc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 -- 172.21.15.107:0/948513878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 msgr2=0x56366bacc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bacc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 -- 172.21.15.107:0/948513878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 msgr2=0x56366bac49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 -- 172.21.15.107:0/948513878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56366b9f44c0 con 0x56366bac4f70 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07afe00640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 0x7f079c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07afe00640 1 -- 172.21.15.107:0/1024191818 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07b0014070 con 0x7f07b8104f70 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07ae000640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 0x7f079c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.303+0000 7f07b6a00640 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f07a8011470 tx=0x7f07a8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6e6f2a70d2535b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07a7e00640 1 -- 172.21.15.107:0/948513878 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8003970 con 0x56366bac4f70 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07a7e00640 1 -- 172.21.15.107:0/948513878 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07a8004020 con 0x56366bac4f70 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07a7e00640 1 -- 172.21.15.107:0/948513878 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800f8c0 con 0x56366bac4f70 2024-03-11T19:52:33.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07c05a2740 1 -- 172.21.15.107:0/948513878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 msgr2=0x56366bac9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07ae000640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 0x7f079c0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006010 tx=0x7f07a002c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.304+0000 7f07c05a2740 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f07a8011470 tx=0x7f07a8011930 comp rx=0 tx=0).stop 2024-03-11T19:52:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 -- 172.21.15.107:0/948513878 shutdown_connections 2024-03-11T19:52:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bacc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bac9a70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 --2- 172.21.15.107:0/948513878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 -- 172.21.15.107:0/948513878 >> 172.21.15.107:0/948513878 conn(0x56366babfaf0 msgr2=0x56366babfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 -- 172.21.15.107:0/948513878 shutdown_connections 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 -- 172.21.15.107:0/948513878 wait complete. 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 -- start start 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.305+0000 7f07c05a2740 1 Processor -- start 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bace8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366b9d1ea0 con 0x56366bac4580 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366b9d1b40 con 0x56366bac4f70 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366b774f20 con 0x56366baca0e0 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07c05a2740 10 client.0 _ll_register_callbacks cb 0x56366b9e5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50474/0 (socket says 172.21.15.107:50474) 2024-03-11T19:52:33.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 -- 172.21.15.107:0/3944062500 learned_addr learned my addr 172.21.15.107:0/3944062500 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b7400640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07ae000640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bace8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 -- 172.21.15.107:0/3944062500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 msgr2=0x56366bad58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 -- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 msgr2=0x56366bace8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bace8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.306+0000 7f07b6a00640 1 -- 172.21.15.107:0/3944062500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8009c40 con 0x56366bac4580 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07b6a00640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07a801b040 tx=0x7f07a8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fc078652b45895 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8004800 con 0x56366bac4580 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07a8003b80 con 0x56366bac4580 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56366ba50c40 con 0x56366bac4580 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a8003d40 con 0x56366bac4580 2024-03-11T19:52:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07c05a2740 10 client.15105 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.307+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56366ba2aec0 con 0x56366bac4580 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56366ba50c40 con 0x56366bac4580 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a80226a0 con 0x56366bac4580 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 10 client.15105 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 20 client.15105 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bada900 0x56366bad9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 4 client.15105 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56366b72c6c0 con 0x56366bada900 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.308+0000 7f07ae000640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bada900 0x56366bad9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.309+0000 7f07ae000640 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bada900 0x56366bad9b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5df9c19ff883bdb server_cookie=dcdcaed3454436ad in_seq=0 out_seq=0 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.309+0000 7f07a4c00640 10 client.15105 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07a4c00640 1 -- 172.21.15.107:0/3944062500 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56366b72c6c0 con 0x56366bada900 2024-03-11T19:52:33.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07a4c00640 10 client.15105 handle_command_reply: tid=0 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 client.15105 shutdown 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 msgr2=0x56366bace370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07a801b040 tx=0x7f07a8004cf0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 shutdown_connections 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bada900 0x56366bad9b60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bace8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3944062500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bace370 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 >> 172.21.15.107:0/3944062500 conn(0x56366babfaf0 msgr2=0x56366babee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07a4c00640 0 client.15105 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:52:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 shutdown_connections 2024-03-11T19:52:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 1 -- 172.21.15.107:0/3944062500 wait complete. 2024-03-11T19:52:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.310+0000 7f07c05a2740 20 client.15105 trim_cache size 0 max 16384 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.327+0000 7f07c05a2740 1 Processor -- start 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.327+0000 7f07c05a2740 1 -- start start 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.327+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bae2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07c05a2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366bada4d0 con 0x56366bac4f70 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07c05a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366badb7f0 con 0x56366bac4580 2024-03-11T19:52:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07c05a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366bad61b0 con 0x56366baca0e0 2024-03-11T19:52:33.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50482/0 (socket says 172.21.15.107:50482) 2024-03-11T19:52:33.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 -- 172.21.15.107:0/3104174855 learned_addr learned my addr 172.21.15.107:0/3104174855 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07b7400640 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bae2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07b6a00640 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 -- 172.21.15.107:0/3104174855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 msgr2=0x56366bae2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bae2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 -- 172.21.15.107:0/3104174855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 msgr2=0x56366bac9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.328+0000 7f07ae000640 1 -- 172.21.15.107:0/3104174855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56366b9c78b0 con 0x56366bac4f70 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.329+0000 7f07ae000640 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009230 tx=0x7f07a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7471b4e2d870324e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.329+0000 7f07a7400640 1 -- 172.21.15.107:0/3104174855 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0056410 con 0x56366bac4f70 2024-03-11T19:52:33.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.329+0000 7f07a7400640 1 -- 172.21.15.107:0/3104174855 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07a0056e90 con 0x56366bac4f70 2024-03-11T19:52:33.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.329+0000 7f07a7400640 1 -- 172.21.15.107:0/3104174855 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0003850 con 0x56366bac4f70 2024-03-11T19:52:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- 172.21.15.107:0/3104174855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 msgr2=0x56366bad5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009230 tx=0x7f07a0009260 comp rx=0 tx=0).stop 2024-03-11T19:52:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- 172.21.15.107:0/3104174855 shutdown_connections 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bae2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4f70 0x56366bad5810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 --2- 172.21.15.107:0/3104174855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4580 0x56366bac9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- 172.21.15.107:0/3104174855 >> 172.21.15.107:0/3104174855 conn(0x56366babfaf0 msgr2=0x56366babfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- 172.21.15.107:0/3104174855 shutdown_connections 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- 172.21.15.107:0/3104174855 wait complete. 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 -- start start 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.330+0000 7f07c05a2740 1 Processor -- start 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bad2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366bb38d10 con 0x56366bac4580 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366bb38e50 con 0x56366bac4f70 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366bb38f90 con 0x56366baca0e0 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07c05a2740 10 client.0 _ll_register_callbacks cb 0x7f07a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50496/0 (socket says 172.21.15.107:50496) 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 -- 172.21.15.107:0/4092979992 learned_addr learned my addr 172.21.15.107:0/4092979992 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07ae000640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bad2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b7400640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 -- 172.21.15.107:0/4092979992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 msgr2=0x56366bad2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 -- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 msgr2=0x56366bad2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bad2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b6a00640 1 -- 172.21.15.107:0/4092979992 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b810c100 con 0x56366bac4580 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07ae000640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bad2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.331+0000 7f07b7400640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07b6a00640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f80 tx=0x7f07a8004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546bdaf5983a48a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011220 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07a8022ea0 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56366badb930 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a8003b80 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07c05a2740 10 client.15111 fetch_fsmap learned FSMap version 23 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.332+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56366b9ed510 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56366badb930 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07a800f760 con 0x56366bac4580 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 10 client.15111 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:52:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 20 client.15111 populate_metadata read hostname 'smithi107' 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bb3df30 0x56366bb3a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 4 client.15111 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56366bada610 con 0x56366bb3df30 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.333+0000 7f07ae000640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bb3df30 0x56366bb3a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07ae000640 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bb3df30 0x56366bb3a3a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185cf4469bdf8de server_cookie=9829136f00d3d2b5 in_seq=0 out_seq=0 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07a4c00640 10 client.15111 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07a4c00640 1 -- 172.21.15.107:0/4092979992 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56366bada610 con 0x56366bb3df30 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07a4c00640 10 client.15111 handle_command_reply: tid=0 2024-03-11T19:52:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 client.15111 shutdown 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 msgr2=0x56366bad58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f80 tx=0x7f07a8004990 comp rx=0 tx=0).stop 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 shutdown_connections 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56366bb3df30 0x56366bb3a3a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56366bac4f70 0x56366bad2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56366baca0e0 0x56366bad2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 --2- 172.21.15.107:0/4092979992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56366bac4580 0x56366bad58f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07a4c00640 0 client.15111 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 >> 172.21.15.107:0/4092979992 conn(0x56366babfaf0 msgr2=0x56366bac9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 shutdown_connections 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 1 -- 172.21.15.107:0/4092979992 wait complete. 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.334+0000 7f07c05a2740 20 client.15111 trim_cache size 0 max 16384 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24590, 2024-03-11T19:52:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 157.16142266400001, 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 163.848698681 2024-03-11T19:52:33.352 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 msgr2=0x7f079c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 0x7f079c0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006010 tx=0x7f07a002c5f0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 msgr2=0x7f07b81a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e970 tx=0x7f07b000ed70 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 shutdown_connections 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f079c0a1a30 0x7f079c0a3ef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b81045b0 0x7f07b8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8104f70 0x7f07b81a0a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 --2- 172.21.15.107:0/1024191818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f07b81033c0 0x7f07b8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 >> 172.21.15.107:0/1024191818 conn(0x7f07b80fe930 msgr2=0x7f07b81078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 shutdown_connections 2024-03-11T19:52:33.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:33.335+0000 7f07bd800640 1 -- 172.21.15.107:0/1024191818 wait complete. 2024-03-11T19:52:33.356 INFO:tasks.cephadm:Setup complete, yielding 2024-03-11T19:52:33.357 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:52:33.366 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:52:33.367 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-11T19:52:33.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:33 smithi145 ceph-mon[36730]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:33.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:33 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2682896686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:33 smithi107 ceph-mon[31712]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:33.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:33 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2682896686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:33.809 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:33 smithi122 ceph-mon[36397]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:33 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2682896686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:34.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:34.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:34.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:34 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:34.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:34.879 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:34.880 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:34 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:34 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.659 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:35 smithi107 ceph-mon[31712]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:35.659 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.659 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.659 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:35.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:35 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:35 smithi122 ceph-mon[36397]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:35 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:35 smithi145 ceph-mon[36730]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:36.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:35 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 -- 172.21.15.107:0/1046155845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 msgr2=0x7f628010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 --2- 172.21.15.107:0/1046155845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f628010c030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6278009f90 tx=0x7f627800b330 comp rx=0 tx=0).stop 2024-03-11T19:52:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 -- 172.21.15.107:0/1046155845 shutdown_connections 2024-03-11T19:52:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 --2- 172.21.15.107:0/1046155845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f628010c030 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 --2- 172.21.15.107:0/1046155845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 0x7f62800740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 --2- 172.21.15.107:0/1046155845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 -- 172.21.15.107:0/1046155845 >> 172.21.15.107:0/1046155845 conn(0x7f62800fd220 msgr2=0x7f62800ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 -- 172.21.15.107:0/1046155845 shutdown_connections 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.172+0000 7f6287e00700 1 -- 172.21.15.107:0/1046155845 wait complete. 2024-03-11T19:52:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.174+0000 7f6287e00700 1 Processor -- start 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.174+0000 7f6287e00700 1 -- start start 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.174+0000 7f6287e00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.174+0000 7f6287e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 0x7f628019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.174+0000 7f6287e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f62801a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6287e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a1ad0 con 0x7f6280075c40 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6287e00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a1c10 con 0x7f6280074670 2024-03-11T19:52:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6287e00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801a1da0 con 0x7f6280109ba0 2024-03-11T19:52:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47068/0 (socket says 172.21.15.107:47068) 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 -- 172.21.15.107:0/3279195380 learned_addr learned my addr 172.21.15.107:0/3279195380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f62801a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6284a00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 0x7f628019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 -- 172.21.15.107:0/3279195380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 msgr2=0x7f62801a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f62801a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 -- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 msgr2=0x7f628019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 0x7f628019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.175+0000 7f6285400700 1 -- 172.21.15.107:0/3279195380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740095f0 con 0x7f6280074670 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f6285400700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f627400ec10 tx=0x7f627400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400cc90 con 0x7f6280074670 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6274004500 con 0x7f6280074670 2024-03-11T19:52:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274010690 con 0x7f6280074670 2024-03-11T19:52:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6278009c40 con 0x7f6280074670 2024-03-11T19:52:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.176+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62801a23a0 con 0x7f6280074670 2024-03-11T19:52:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.178+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62801089b0 con 0x7f6280074670 2024-03-11T19:52:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.179+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6274003710 con 0x7f6280074670 2024-03-11T19:52:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.181+0000 7f627e000700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 0x7f62680a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.181+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6274014070 con 0x7f6280074670 2024-03-11T19:52:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.181+0000 7f6284a00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 0x7f62680a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.182+0000 7f6284a00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 0x7f62680a4000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f626c009c60 tx=0x7f626c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.185+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6274096a00 con 0x7f6280074670 2024-03-11T19:52:36.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.447+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f628002d8a0 con 0x7f62680a1b40 2024-03-11T19:52:36.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.450+0000 7f627e000700 1 -- 172.21.15.107:0/3279195380 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f628002d8a0 con 0x7f62680a1b40 2024-03-11T19:52:36.454 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-03-11T19:52:36.454 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-03-11T19:52:36.455 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-03-11T19:52:36.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 msgr2=0x7f62680a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:36.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 0x7f62680a4000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f626c009c60 tx=0x7f626c009260 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 msgr2=0x7f6280199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f627400ec10 tx=0x7f627400ef20 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 shutdown_connections 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f62680a1b40 0x7f62680a4000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6280109ba0 0x7f62801a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6280074670 0x7f6280199f40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 --2- 172.21.15.107:0/3279195380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6280075c40 0x7f628019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.452+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 >> 172.21.15.107:0/3279195380 conn(0x7f62800fd220 msgr2=0x7f6280105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.453+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 shutdown_connections 2024-03-11T19:52:36.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:36.453+0000 7f6287e00700 1 -- 172.21.15.107:0/3279195380 wait complete. 2024-03-11T19:52:37.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:36 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:36 smithi107 ceph-mon[31712]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:37.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:36 smithi107 ceph-mon[31712]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:37.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:36 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:36 smithi122 ceph-mon[36397]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:37.263 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:36 smithi122 ceph-mon[36397]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:37.287 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:36 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:36 smithi145 ceph-mon[36730]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:37.288 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:36 smithi145 ceph-mon[36730]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:38.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-11T19:52:38.445 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:39.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:39 smithi107 ceph-mon[31712]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:39.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:39 smithi122 ceph-mon[36397]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:39.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:39 smithi145 ceph-mon[36730]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 -- 172.21.15.107:0/315443405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48101cc0 msgr2=0x7f7d481020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/315443405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48101cc0 0x7f7d481020e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d380099c0 tx=0x7f7d38009cd0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 -- 172.21.15.107:0/315443405 shutdown_connections 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/315443405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48103860 0x7f7d4810a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/315443405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48101cc0 0x7f7d481020e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/315443405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48102ec0 0x7f7d48103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 -- 172.21.15.107:0/315443405 >> 172.21.15.107:0/315443405 conn(0x7f7d480fd220 msgr2=0x7f7d480ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 -- 172.21.15.107:0/315443405 shutdown_connections 2024-03-11T19:52:40.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.496+0000 7f7d4cc00700 1 -- 172.21.15.107:0/315443405 wait complete. 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.497+0000 7f7d4cc00700 1 Processor -- start 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.497+0000 7f7d4cc00700 1 -- start start 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48102ec0 0x7f7d4819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 0x7f7d481a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a1a20 con 0x7f7d48103860 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a1b60 con 0x7f7d48102ec0 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d4cc00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d481a1cf0 con 0x7f7d48101cc0 2024-03-11T19:52:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:44366/0 (socket says 172.21.15.107:44366) 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 -- 172.21.15.107:0/1962194540 learned_addr learned my addr 172.21.15.107:0/1962194540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d46800700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 0x7f7d481a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45400700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48102ec0 0x7f7d4819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 -- 172.21.15.107:0/1962194540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48102ec0 msgr2=0x7f7d4819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48102ec0 0x7f7d4819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 -- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 msgr2=0x7f7d481a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 0x7f7d481a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d45e00700 1 -- 172.21.15.107:0/1962194540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d48101cc0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.498+0000 7f7d46800700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 0x7f7d481a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d45e00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009fd0 tx=0x7f7d380056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x7f7d48101cc0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d481a1f70 con 0x7f7d48101cc0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d481a2320 con 0x7f7d48101cc0 2024-03-11T19:52:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d3800be80 con 0x7f7d48101cc0 2024-03-11T19:52:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.499+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800fe40 con 0x7f7d48101cc0 2024-03-11T19:52:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.500+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d48068930 con 0x7f7d48101cc0 2024-03-11T19:52:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.504+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7d3800f460 con 0x7f7d48101cc0 2024-03-11T19:52:40.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.505+0000 7f7d3ea00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 0x7f7d2c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:40.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.505+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7d380ce7c0 con 0x7f7d48101cc0 2024-03-11T19:52:40.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.506+0000 7f7d45400700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 0x7f7d2c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.506+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d38097650 con 0x7f7d48101cc0 2024-03-11T19:52:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.506+0000 7f7d45400700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 0x7f7d2c0a4000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009bf0 tx=0x7f7d34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:40.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.741+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d480755c0 con 0x7f7d2c0a1b40 2024-03-11T19:52:40.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:40.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:40 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:40.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.772+0000 7f7d3ea00700 1 -- 172.21.15.107:0/1962194540 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f7d480755c0 con 0x7f7d2c0a1b40 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (2m) 1s ago 2m 18.6M - 19.0.0-1607-g94a53970 a6477a474d71 9790c706d881 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi122 running (2m) 4s ago 2m 27.6M - 19.0.0-1607-g94a53970 a6477a474d71 805b7f0c5678 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi145 running (3m) 5s ago 3m 21.7M - 19.0.0-1607-g94a53970 a6477a474d71 f8b8faa91594 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (3m) 1s ago 3m 13.2M - 19.0.0-1607-g94a53970 a6477a474d71 925e0b2b2f4e 2024-03-11T19:52:40.777 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi122 running (3m) 4s ago 3m 16.1M - 19.0.0-1607-g94a53970 a6477a474d71 1c9c0e4c0a66 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi145 running (3m) 5s ago 3m 20.5M - 19.0.0-1607-g94a53970 a6477a474d71 965f7b1c5428 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (2m) 1s ago 2m 13.0M - 19.0.0-1607-g94a53970 a6477a474d71 369a10c25195 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi122 running (3m) 4s ago 3m 21.7M - 19.0.0-1607-g94a53970 a6477a474d71 6a0dd0caeea4 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi145 running (3m) 5s ago 3m 20.0M - 19.0.0-1607-g94a53970 a6477a474d71 c396f7974a26 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (3m) 1s ago 3m 13.2M - 19.0.0-1607-g94a53970 a6477a474d71 027681fb6c77 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi122 running (3m) 4s ago 3m 14.8M - 19.0.0-1607-g94a53970 a6477a474d71 aab78686795d 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi145 running (3m) 5s ago 3m 14.8M - 19.0.0-1607-g94a53970 a6477a474d71 b159133aaa2b 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (11m) 1s ago 11m 515M - 19.0.0-1607-g94a53970 a6477a474d71 6d84723511de 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi122 *:8443,8765 running (9m) 4s ago 9m 454M - 19.0.0-1607-g94a53970 a6477a474d71 0b1db61399b1 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi145 *:8443,8765 running (9m) 5s ago 9m 452M - 19.0.0-1607-g94a53970 a6477a474d71 9847e0d8990b 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (11m) 1s ago 11m 53.5M 2048M 19.0.0-1607-g94a53970 a6477a474d71 d5b6eaa81c1e 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi122 running (9m) 4s ago 9m 43.9M 2048M 19.0.0-1607-g94a53970 a6477a474d71 68ea3625f0df 2024-03-11T19:52:40.778 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi145 running (9m) 5s ago 9m 43.1M 2048M 19.0.0-1607-g94a53970 a6477a474d71 e60cd6454fc4 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 1s ago 8m 56.7M 4272M 19.0.0-1607-g94a53970 a6477a474d71 4e96d7071606 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi122 running (8m) 4s ago 8m 61.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 55ad9269210c 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi145 running (8m) 5s ago 8m 48.3M 4270M 19.0.0-1607-g94a53970 a6477a474d71 38fb4eddbd68 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (7m) 1s ago 7m 44.7M 4272M 19.0.0-1607-g94a53970 a6477a474d71 f3dc6f8b014d 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi122 running (7m) 4s ago 7m 44.2M 4270M 19.0.0-1607-g94a53970 a6477a474d71 60445d508c9d 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi145 running (6m) 5s ago 6m 60.1M 4270M 19.0.0-1607-g94a53970 a6477a474d71 bab56d11112b 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (6m) 1s ago 6m 43.0M 4272M 19.0.0-1607-g94a53970 a6477a474d71 e056f7978a1a 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi122 running (5m) 4s ago 5m 46.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 cc01bcc8e771 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi145 running (5m) 5s ago 5m 47.1M 4270M 19.0.0-1607-g94a53970 a6477a474d71 d38e042dcce4 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (4m) 1s ago 4m 42.6M 4272M 19.0.0-1607-g94a53970 a6477a474d71 c50065bc188d 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi122 running (4m) 4s ago 4m 56.6M 4270M 19.0.0-1607-g94a53970 a6477a474d71 23328961a885 2024-03-11T19:52:40.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi145 running (3m) 5s ago 3m 43.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 c95738bc1e60 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 msgr2=0x7f7d2c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 0x7f7d2c0a4000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009bf0 tx=0x7f7d34009260 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 msgr2=0x7f7d48199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009fd0 tx=0x7f7d380056a0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 shutdown_connections 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7d2c0a1b40 0x7f7d2c0a4000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103860 0x7f7d481a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7d48101cc0 0x7f7d48199e90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 --2- 172.21.15.107:0/1962194540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d48102ec0 0x7f7d4819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:40.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 >> 172.21.15.107:0/1962194540 conn(0x7f7d480fd220 msgr2=0x7f7d48073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:40.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 shutdown_connections 2024-03-11T19:52:40.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:40.777+0000 7f7d4cc00700 1 -- 172.21.15.107:0/1962194540 wait complete. 2024-03-11T19:52:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:40.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:40 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:41.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:41.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:40 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:41.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-11T19:52:41.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:41 smithi145 ceph-mon[36730]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:41.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:41.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:41.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:41 smithi145 ceph-mon[36730]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:41.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:41 smithi107 ceph-mon[31712]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:41 smithi107 ceph-mon[31712]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:41.981 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:42.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:41 smithi122 ceph-mon[36397]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:42.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:42.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:42.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:41 smithi122 ceph-mon[36397]: from='mgr.14152 172.21.15.107:0/2194081219' entity='mgr.x' 2024-03-11T19:52:42.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:42 smithi145 ceph-mon[36730]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:42.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:42 smithi107 ceph-mon[31712]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:43.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:42 smithi122 ceph-mon[36397]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:43.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:43 smithi145 ceph-mon[36730]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:43.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:43 smithi107 ceph-mon[31712]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:44.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:43 smithi122 ceph-mon[36397]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.371+0000 7f6056600700 1 -- 172.21.15.107:0/1750273760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60500746d0 msgr2=0x7f6050074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.371+0000 7f6056600700 1 --2- 172.21.15.107:0/1750273760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60500746d0 0x7f6050074b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60380098f0 tx=0x7f6038009c00 comp rx=0 tx=0).stop 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 -- 172.21.15.107:0/1750273760 shutdown_connections 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 --2- 172.21.15.107:0/1750273760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f605010e290 0x7f6050110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 --2- 172.21.15.107:0/1750273760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6050075ca0 0x7f6050074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 --2- 172.21.15.107:0/1750273760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f60500746d0 0x7f6050074b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 -- 172.21.15.107:0/1750273760 >> 172.21.15.107:0/1750273760 conn(0x7f60500fd210 msgr2=0x7f60500ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 -- 172.21.15.107:0/1750273760 shutdown_connections 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.372+0000 7f6056600700 1 -- 172.21.15.107:0/1750273760 wait complete. 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.373+0000 7f6056600700 1 Processor -- start 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.373+0000 7f6056600700 1 -- start start 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.373+0000 7f6056600700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60500746d0 0x7f605019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6056600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6056600700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f605010e290 0x7f60501a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6056600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a5eb0 con 0x7f6050075ca0 2024-03-11T19:52:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6056600700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a5ff0 con 0x7f60500746d0 2024-03-11T19:52:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6056600700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a6180 con 0x7f605010e290 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59798/0 (socket says 172.21.15.107:59798) 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 -- 172.21.15.107:0/1810546933 learned_addr learned my addr 172.21.15.107:0/1810546933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f604f400700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60500746d0 0x7f605019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f604fe00700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f605010e290 0x7f60501a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 -- 172.21.15.107:0/1810546933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f605010e290 msgr2=0x7f60501a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f605010e290 0x7f60501a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 -- 172.21.15.107:0/1810546933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60500746d0 msgr2=0x7f605019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60500746d0 0x7f605019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.374+0000 7f6047e00700 1 -- 172.21.15.107:0/1810546933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048009d00 con 0x7f6050075ca0 2024-03-11T19:52:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f6047e00700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f603800bf50 tx=0x7f6038005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603801c070 con 0x7f6050075ca0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6038009520 con 0x7f6050075ca0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f603800ba40 con 0x7f6050075ca0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038003dc0 con 0x7f6050075ca0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.375+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501a66e0 con 0x7f6050075ca0 2024-03-11T19:52:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.377+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60501044f0 con 0x7f6050075ca0 2024-03-11T19:52:44.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.379+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f603800b560 con 0x7f6050075ca0 2024-03-11T19:52:44.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.380+0000 7f604cc00700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 0x7f60340a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.380+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f60380ce8b0 con 0x7f6050075ca0 2024-03-11T19:52:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.380+0000 7f604f400700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 0x7f60340a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.381+0000 7f604f400700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 0x7f60340a4000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6050076080 tx=0x7f6040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.383+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6038097740 con 0x7f6050075ca0 2024-03-11T19:52:44.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.617+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f605002d7d0 con 0x7f60340a1b40 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.625+0000 7f604cc00700 1 -- 172.21.15.107:0/1810546933 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f605002d7d0 con 0x7f60340a1b40 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 9s ago 3m smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi122=b;smithi122=e;smithi122=h;smithi122=k;smithi145=c;smithi145=f;smithi145=i;smithi145=l;count:12 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 9s ago 9m smithi107=x;smithi122=y;smithi145=z;count:3 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 9s ago 9m smithi107:172.21.15.107=a;smithi122:172.21.15.122=b;smithi145:172.21.15.145=c;count:3 2024-03-11T19:52:44.629 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 9s ago - 2024-03-11T19:52:44.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 msgr2=0x7f60340a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:44.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 0x7f60340a4000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6050076080 tx=0x7f6040009260 comp rx=0 tx=0).stop 2024-03-11T19:52:44.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 msgr2=0x7f605019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:44.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f603800bf50 tx=0x7f6038005250 comp rx=0 tx=0).stop 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 shutdown_connections 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f60340a1b40 0x7f60340a4000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f605010e290 0x7f60501a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60500746d0 0x7f605019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 --2- 172.21.15.107:0/1810546933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6050075ca0 0x7f605019e860 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.628+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 >> 172.21.15.107:0/1810546933 conn(0x7f60500fd210 msgr2=0x7f6050101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.629+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 shutdown_connections 2024-03-11T19:52:44.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:44.629+0000 7f6056600700 1 -- 172.21.15.107:0/1810546933 wait complete. 2024-03-11T19:52:45.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-11T19:52:45.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:45 smithi107 ceph-mon[31712]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:45.890 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:46.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:45 smithi122 ceph-mon[36397]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:46.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:45 smithi145 ceph-mon[36730]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:46.933 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:46 smithi107 ceph-mon[31712]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:47.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:46 smithi122 ceph-mon[36397]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:47.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:46 smithi145 ceph-mon[36730]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:47.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:47 smithi107 ceph-mon[31712]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:47.914 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:47 smithi122 ceph-mon[36397]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:48.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:47 smithi145 ceph-mon[36730]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.278+0000 7f7d03800700 1 -- 172.21.15.107:0/782821056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc109ba0 msgr2=0x7f7cfc10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.278+0000 7f7d03800700 1 --2- 172.21.15.107:0/782821056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc10c010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400b0a0 tx=0x7f7cf400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 -- 172.21.15.107:0/782821056 shutdown_connections 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 --2- 172.21.15.107:0/782821056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc10c010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 --2- 172.21.15.107:0/782821056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc075c20 0x7f7cfc074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 --2- 172.21.15.107:0/782821056 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 0x7f7cfc074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 -- 172.21.15.107:0/782821056 >> 172.21.15.107:0/782821056 conn(0x7f7cfc0fd280 msgr2=0x7f7cfc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 -- 172.21.15.107:0/782821056 shutdown_connections 2024-03-11T19:52:48.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.279+0000 7f7d03800700 1 -- 172.21.15.107:0/782821056 wait complete. 2024-03-11T19:52:48.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.280+0000 7f7d03800700 1 Processor -- start 2024-03-11T19:52:48.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.280+0000 7f7d03800700 1 -- start start 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 0x7f7cfc199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc075c20 0x7f7cfc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1a1ac0 con 0x7f7cfc075c20 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1a1c00 con 0x7f7cfc109ba0 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d03800700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1a1d90 con 0x7f7cfc074650 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7cfbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc075c20 0x7f7cfc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:48.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:55296/0 (socket says 172.21.15.107:55296) 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 -- 172.21.15.107:0/3664260416 learned_addr learned my addr 172.21.15.107:0/3664260416 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d00e00700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 0x7f7cfc199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 -- 172.21.15.107:0/3664260416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 msgr2=0x7f7cfc199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 0x7f7cfc199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 -- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc075c20 msgr2=0x7f7cfc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc075c20 0x7f7cfc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.281+0000 7f7d01800700 1 -- 172.21.15.107:0/3664260416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009d00 con 0x7f7cfc109ba0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7d01800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4007ab0 tx=0x7f7cf400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400e040 con 0x7f7cfc109ba0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc1a2010 con 0x7f7cfc109ba0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf4004e60 con 0x7f7cfc109ba0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40052a0 con 0x7f7cfc109ba0 2024-03-11T19:52:48.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.282+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc1a24d0 con 0x7f7cfc109ba0 2024-03-11T19:52:48.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.283+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc1089b0 con 0x7f7cfc109ba0 2024-03-11T19:52:48.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.285+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7cf4005400 con 0x7f7cfc109ba0 2024-03-11T19:52:48.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.286+0000 7f7cf9600700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 0x7f7ce40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:48.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.287+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7cf40ced40 con 0x7f7cfc109ba0 2024-03-11T19:52:48.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.287+0000 7f7d00e00700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 0x7f7ce40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:48.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.287+0000 7f7d00e00700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 0x7f7ce40a4000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009c60 tx=0x7f7cf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:48.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.289+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cf4097bd0 con 0x7f7cfc109ba0 2024-03-11T19:52:48.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.506+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cfc02d870 con 0x7f7ce40a1b40 2024-03-11T19:52:48.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.508+0000 7f7cf9600700 1 -- 172.21.15.107:0/3664260416 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f7cfc02d870 con 0x7f7ce40a1b40 2024-03-11T19:52:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-03-11T19:52:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-03-11T19:52:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:smithi122 172.21.15.122 2024-03-11T19:52:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:smithi145 172.21.15.145 2024-03-11T19:52:48.513 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2024-03-11T19:52:48.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.511+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 msgr2=0x7f7ce40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:48.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.511+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 0x7f7ce40a4000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009c60 tx=0x7f7cf0009260 comp rx=0 tx=0).stop 2024-03-11T19:52:48.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.511+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 msgr2=0x7f7cfc1a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:48.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.511+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4007ab0 tx=0x7f7cf400bce0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 shutdown_connections 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7ce40a1b40 0x7f7ce40a4000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cfc109ba0 0x7f7cfc1a14f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7cfc074650 0x7f7cfc199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 --2- 172.21.15.107:0/3664260416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cfc075c20 0x7f7cfc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 >> 172.21.15.107:0/3664260416 conn(0x7f7cfc0fd280 msgr2=0x7f7cfc105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 shutdown_connections 2024-03-11T19:52:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:48.512+0000 7f7d03800700 1 -- 172.21.15.107:0/3664260416 wait complete. 2024-03-11T19:52:48.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:48 smithi107 ceph-mon[31712]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:48.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:48 smithi107 ceph-mon[31712]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:49.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:48 smithi122 ceph-mon[36397]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:49.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:48 smithi122 ceph-mon[36397]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:49.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:48 smithi145 ceph-mon[36730]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:49.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:48 smithi145 ceph-mon[36730]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:49.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-11T19:52:49.884 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:51.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:51 smithi107 ceph-mon[31712]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:51.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:51 smithi122 ceph-mon[36397]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:51.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:51 smithi145 ceph-mon[36730]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 -- 172.21.15.107:0/184137361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 msgr2=0x7fd710109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 --2- 172.21.15.107:0/184137361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd710109950 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80099c0 tx=0x7fd6f8009cd0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 -- 172.21.15.107:0/184137361 shutdown_connections 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 --2- 172.21.15.107:0/184137361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd710109e90 0x7fd710110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 --2- 172.21.15.107:0/184137361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7101082f0 0x7fd710108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 --2- 172.21.15.107:0/184137361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd710109950 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 -- 172.21.15.107:0/184137361 >> 172.21.15.107:0/184137361 conn(0x7fd7100772c0 msgr2=0x7fd710079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:52.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 -- 172.21.15.107:0/184137361 shutdown_connections 2024-03-11T19:52:52.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.286+0000 7fd717a00700 1 -- 172.21.15.107:0/184137361 wait complete. 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.287+0000 7fd717a00700 1 Processor -- start 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.287+0000 7fd717a00700 1 -- start start 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 0x7fd71019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd71019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 0x7fd7101a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a5dd0 con 0x7fd710109e90 2024-03-11T19:52:52.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a5f10 con 0x7fd7101094f0 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd717a00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7101a60a0 con 0x7fd7101082f0 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 0x7fd71019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 0x7fd7101a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 0x7fd71019e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35002/0 (socket says 172.21.15.107:35002) 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715000700 1 -- 172.21.15.107:0/3023815113 learned_addr learned my addr 172.21.15.107:0/3023815113 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd707e00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd71019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 -- 172.21.15.107:0/3023815113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 msgr2=0x7fd71019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 0x7fd71019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 -- 172.21.15.107:0/3023815113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 msgr2=0x7fd71019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd71019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.288+0000 7fd715a00700 1 -- 172.21.15.107:0/3023815113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80095f0 con 0x7fd710109e90 2024-03-11T19:52:52.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd715a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 0x7fd7101a5800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e9a0 tx=0x7fd70800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:52.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708004220 con 0x7fd710109e90 2024-03-11T19:52:52.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd708004d10 con 0x7fd710109e90 2024-03-11T19:52:52.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708010700 con 0x7fd710109e90 2024-03-11T19:52:52.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7101a6380 con 0x7fd710109e90 2024-03-11T19:52:52.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.289+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7101a68d0 con 0x7fd710109e90 2024-03-11T19:52:52.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.290+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71010f650 con 0x7fd710109e90 2024-03-11T19:52:52.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.292+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fd708004750 con 0x7fd710109e90 2024-03-11T19:52:52.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.294+0000 7fd70e000700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 0x7fd6f40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:52.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.294+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd708014070 con 0x7fd710109e90 2024-03-11T19:52:52.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.294+0000 7fd715000700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 0x7fd6f40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:52.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.295+0000 7fd715000700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 0x7fd6f40a4000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd70000f5e0 tx=0x7fd70000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:52.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.297+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7080963e0 con 0x7fd710109e90 2024-03-11T19:52:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.526+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd71002d870 con 0x7fd6f40a1b40 2024-03-11T19:52:52.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.530+0000 7fd70e000700 1 -- 172.21.15.107:0/3023815113 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fd71002d870 con 0x7fd6f40a1b40 2024-03-11T19:52:52.534 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-11T19:52:52.535 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-11T19:52:52.535 INFO:teuthology.orchestra.run.smithi107.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-11T19:52:52.535 INFO:teuthology.orchestra.run.smithi107.stdout:smithi145 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N9400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-11T19:52:52.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 msgr2=0x7fd6f40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:52.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 0x7fd6f40a4000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd70000f5e0 tx=0x7fd70000b040 comp rx=0 tx=0).stop 2024-03-11T19:52:52.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 msgr2=0x7fd7101a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:52.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 0x7fd7101a5800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e9a0 tx=0x7fd70800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 shutdown_connections 2024-03-11T19:52:52.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fd6f40a1b40 0x7fd6f40a4000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd710109e90 0x7fd7101a5800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd7101082f0 0x7fd71019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 --2- 172.21.15.107:0/3023815113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7101094f0 0x7fd71019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.533+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 >> 172.21.15.107:0/3023815113 conn(0x7fd7100772c0 msgr2=0x7fd71010c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.534+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 shutdown_connections 2024-03-11T19:52:52.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:52.534+0000 7fd717a00700 1 -- 172.21.15.107:0/3023815113 wait complete. 2024-03-11T19:52:53.452 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:52:53.460 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:52:53.461 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-11T19:52:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:53 smithi122 ceph-mon[36397]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:53.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:53 smithi122 ceph-mon[36397]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:53 smithi145 ceph-mon[36730]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:53.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:53 smithi145 ceph-mon[36730]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:53 smithi107 ceph-mon[31712]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:52:53.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:53 smithi107 ceph-mon[31712]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:52:53.908 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:55.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:55 smithi107 ceph-mon[31712]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:55.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:55 smithi122 ceph-mon[36397]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:55.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:55 smithi145 ceph-mon[36730]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:56.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 -- 172.21.15.107:0/4194804338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 msgr2=0x7ff880074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 --2- 172.21.15.107:0/4194804338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff880074b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff868009990 tx=0x7ff868009ca0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 -- 172.21.15.107:0/4194804338 shutdown_connections 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 --2- 172.21.15.107:0/4194804338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff88010e1a0 0x7ff880110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 --2- 172.21.15.107:0/4194804338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff880075ca0 0x7ff880074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 --2- 172.21.15.107:0/4194804338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff880074b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.107+0000 7ff87fe00700 1 -- 172.21.15.107:0/4194804338 >> 172.21.15.107:0/4194804338 conn(0x7ff8800fd180 msgr2=0x7ff8800ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.108+0000 7ff87fe00700 1 -- 172.21.15.107:0/4194804338 shutdown_connections 2024-03-11T19:52:56.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.108+0000 7ff87fe00700 1 -- 172.21.15.107:0/4194804338 wait complete. 2024-03-11T19:52:56.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 Processor -- start 2024-03-11T19:52:56.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 -- start start 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff880075ca0 0x7ff88019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88010e1a0 0x7ff8801a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8801a5d30 con 0x7ff880075ca0 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.109+0000 7ff87fe00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8801a5e70 con 0x7ff8800746d0 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87fe00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8801a6000 con 0x7ff88010e1a0 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87f400700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88010e1a0 0x7ff8801a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:54556/0 (socket says 172.21.15.107:54556) 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 -- 172.21.15.107:0/2775298781 learned_addr learned my addr 172.21.15.107:0/2775298781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:52:56.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff877e00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff880075ca0 0x7ff88019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:56.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 -- 172.21.15.107:0/2775298781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88010e1a0 msgr2=0x7ff8801a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88010e1a0 0x7ff8801a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 -- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff880075ca0 msgr2=0x7ff88019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff880075ca0 0x7ff88019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.110+0000 7ff87ea00700 1 -- 172.21.15.107:0/2775298781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8680095f0 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff87ea00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d950 tx=0x7ff87000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870004a30 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8801a62e0 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff870005500 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87000fc80 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.111+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8801a67e0 con 0x7ff8800746d0 2024-03-11T19:52:56.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.112+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff880104400 con 0x7ff8800746d0 2024-03-11T19:52:56.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.114+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff870005000 con 0x7ff8800746d0 2024-03-11T19:52:56.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.115+0000 7ff877400700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 0x7ff8640a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:56.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.116+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff8700cd3b0 con 0x7ff8800746d0 2024-03-11T19:52:56.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.116+0000 7ff877e00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 0x7ff8640a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:56.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.116+0000 7ff877e00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 0x7ff8640a4000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b650 tx=0x7ff868005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:56.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.118+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff870096230 con 0x7ff8800746d0 2024-03-11T19:52:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.388+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff880113bc0 con 0x7ff8800746d0 2024-03-11T19:52:56.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.389+0000 7ff877400700 1 -- 172.21.15.107:0/2775298781 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2953 (secure 0 0 0) 0x7ff870017020 con 0x7ff8800746d0 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:e23 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:56.394 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 23 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-11T19:49:55.178113+0000 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-11T19:50:29.435340+0000 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-11T19:52:56.395 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 3 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24590,1=24583,2=24547} 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3,4] 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-11T19:52:56.396 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:24590} state up:active seq 3 addr [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{0:24533} state up:standby-replay seq 1 addr [v2:172.21.15.122:6836/547863696,v1:172.21.15.122:6837/547863696] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{1:24583} state up:active seq 12 addr [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{1:24529} state up:standby-replay seq 1 addr [v2:172.21.15.145:6836/2305866182,v1:172.21.15.145:6837/2305866182] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{2:24547} state up:active seq 14 addr [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{2:24517} state up:standby-replay seq 1 addr [v2:172.21.15.145:6834/3941680931,v1:172.21.15.145:6835/3941680931] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.107:6836/3292957465,v1:172.21.15.107:6837/3292957465] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{-1:14451} state up:standby seq 1 addr [v2:172.21.15.107:6838/988917516,v1:172.21.15.107:6839/988917516] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.145:6832/1918081210,v1:172.21.15.145:6833/1918081210] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.397 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{-1:24455} state up:standby seq 1 addr [v2:172.21.15.122:6832/1820097321,v1:172.21.15.122:6833/1820097321] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:24478} state up:standby seq 1 addr [v2:172.21.15.107:6834/1132278897,v1:172.21.15.107:6835/1132278897] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.398 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{-1:24503} state up:standby seq 1 addr [v2:172.21.15.122:6834/1916658439,v1:172.21.15.122:6835/1916658439] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:52:56.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 msgr2=0x7ff8640a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:56.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 0x7ff8640a4000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b650 tx=0x7ff868005fb0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 msgr2=0x7ff88019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:56.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff87000d950 tx=0x7ff87000dc60 comp rx=0 tx=0).stop 2024-03-11T19:52:56.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 shutdown_connections 2024-03-11T19:52:56.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8640a1b40 0x7ff8640a4000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88010e1a0 0x7ff8801a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8800746d0 0x7ff88019e1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 --2- 172.21.15.107:0/2775298781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff880075ca0 0x7ff88019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 >> 172.21.15.107:0/2775298781 conn(0x7ff8800fd180 msgr2=0x7ff880101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 shutdown_connections 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:52:56.392+0000 7ff87fe00700 1 -- 172.21.15.107:0/2775298781 wait complete. 2024-03-11T19:52:56.400 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:52:56.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:56 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2775298781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:52:56.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:56 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2775298781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:52:56.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:56 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2775298781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:52:57.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-11T19:52:57.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:57 smithi122 ceph-mon[36397]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:57.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:57 smithi145 ceph-mon[36730]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:57.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:52:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:57 smithi107 ceph-mon[31712]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:52:59.744 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:52:59 smithi107 ceph-mon[31712]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:59.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:52:59 smithi122 ceph-mon[36397]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:52:59.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:52:59 smithi145 ceph-mon[36730]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:00.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 -- 172.21.15.107:0/4286490817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754074650 msgr2=0x7fc754074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 --2- 172.21.15.107:0/4286490817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754074650 0x7fc754074ad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc740009990 tx=0x7fc740009ca0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 -- 172.21.15.107:0/4286490817 shutdown_connections 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 --2- 172.21.15.107:0/4286490817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754109ba0 0x7fc75410c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 --2- 172.21.15.107:0/4286490817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc754074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 --2- 172.21.15.107:0/4286490817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754074650 0x7fc754074ad0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 -- 172.21.15.107:0/4286490817 >> 172.21.15.107:0/4286490817 conn(0x7fc7540fd280 msgr2=0x7fc7540ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 -- 172.21.15.107:0/4286490817 shutdown_connections 2024-03-11T19:53:00.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.053+0000 7fc759c00700 1 -- 172.21.15.107:0/4286490817 wait complete. 2024-03-11T19:53:00.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 Processor -- start 2024-03-11T19:53:00.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 -- start start 2024-03-11T19:53:00.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754074650 0x7fc754199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:00.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:00.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754109ba0 0x7fc7541a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a19f0 con 0x7fc754109ba0 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a1b30 con 0x7fc754075c20 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.055+0000 7fc759c00700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a1cc0 con 0x7fc754074650 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:54574/0 (socket says 172.21.15.107:54574) 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 -- 172.21.15.107:0/2947454299 learned_addr learned my addr 172.21.15.107:0/2947454299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc753400700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754109ba0 0x7fc7541a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:00.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752a00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754074650 0x7fc754199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 -- 172.21.15.107:0/2947454299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754074650 msgr2=0x7fc754199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754074650 0x7fc754199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 -- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754109ba0 msgr2=0x7fc7541a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754109ba0 0x7fc7541a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 -- 172.21.15.107:0/2947454299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400095f0 con 0x7fc754075c20 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.056+0000 7fc752000700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc7400095c0 tx=0x7fc74000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.057+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74001d070 con 0x7fc754075c20 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.057+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc740005050 con 0x7fc754075c20 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.057+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740004bb0 con 0x7fc754075c20 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.057+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7541a1f40 con 0x7fc754075c20 2024-03-11T19:53:00.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.057+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7541a22f0 con 0x7fc754075c20 2024-03-11T19:53:00.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.058+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7541089b0 con 0x7fc754075c20 2024-03-11T19:53:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.060+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fc740004d10 con 0x7fc754075c20 2024-03-11T19:53:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.061+0000 7fc74b400700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 0x7fc7380a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.061+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc7400ceaf0 con 0x7fc754075c20 2024-03-11T19:53:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.062+0000 7fc752a00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 0x7fc7380a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.062+0000 7fc752a00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 0x7fc7380a3f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc744005950 tx=0x7fc74400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.064+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc740097980 con 0x7fc754075c20 2024-03-11T19:53:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.282+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc75410f8e0 con 0x7fc754075c20 2024-03-11T19:53:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.283+0000 7fc74b400700 1 -- 172.21.15.107:0/2947454299 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5748 (secure 0 0 0) 0x7fc74009b040 con 0x7fc754075c20 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 95 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-11T19:40:45.251508+0000 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-11T19:51:56.516846+0000 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 27 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-03-11T19:53:00.290 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release squid 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/2497501905,v1:172.21.15.107:6803/2497501905] [v2:172.21.15.107:6804/2497501905,v1:172.21.15.107:6805/2497501905] exists,up aaa69835-f991-4647-8793-70e0f2472ebb 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/4246700402,v1:172.21.15.122:6801/4246700402] [v2:172.21.15.122:6802/4246700402,v1:172.21.15.122:6803/4246700402] exists,up 4de9ffc7-d837-48bc-b205-34fe05f25eab 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6800/3949304932,v1:172.21.15.145:6801/3949304932] [v2:172.21.15.145:6802/3949304932,v1:172.21.15.145:6803/3949304932] exists,up 32cbc8a5-cf58-48bc-8e6e-013627c03c26 2024-03-11T19:53:00.291 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/3455904746,v1:172.21.15.107:6811/3455904746] [v2:172.21.15.107:6812/3455904746,v1:172.21.15.107:6813/3455904746] exists,up fec9056e-4b8e-477a-8004-9f802d9b3808 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/1080212908,v1:172.21.15.122:6809/1080212908] [v2:172.21.15.122:6810/1080212908,v1:172.21.15.122:6811/1080212908] exists,up aafc0f8b-5df2-4a3e-86de-88558c60282e 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 38 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6808/3574999361,v1:172.21.15.145:6809/3574999361] [v2:172.21.15.145:6810/3574999361,v1:172.21.15.145:6811/3574999361] exists,up 3144ef43-abab-4f02-b28e-6ac842870975 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 46 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/1993322562,v1:172.21.15.107:6819/1993322562] [v2:172.21.15.107:6820/1993322562,v1:172.21.15.107:6821/1993322562] exists,up 474c8a3c-29ec-45d8-8580-952d3bce081a 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/3856508746,v1:172.21.15.122:6817/3856508746] [v2:172.21.15.122:6818/3856508746,v1:172.21.15.122:6819/3856508746] exists,up 2ddf2101-a45c-448e-8caa-fab9b4d3bb2f 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6816/4190884863,v1:172.21.15.145:6817/4190884863] [v2:172.21.15.145:6818/4190884863,v1:172.21.15.145:6819/4190884863] exists,up e0816612-8f19-48fe-9290-bebea9bcb91d 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 67 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/3311766141,v1:172.21.15.107:6827/3311766141] [v2:172.21.15.107:6828/3311766141,v1:172.21.15.107:6829/3311766141] exists,up cc8774d5-15de-4134-82a4-fdeb3ff19f2a 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 72 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/1130634331,v1:172.21.15.122:6825/1130634331] [v2:172.21.15.122:6826/1130634331,v1:172.21.15.122:6827/1130634331] exists,up 73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 80 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.145:6824/2544565646,v1:172.21.15.145:6825/2544565646] [v2:172.21.15.145:6826/2544565646,v1:172.21.15.145:6827/2544565646] exists,up 007f1865-b782-4f5e-98f1-b358a65840f8 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.5 [8,11] 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.10 [2,1] 2024-03-11T19:53:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.1b [2,4] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.1d [8,11] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.11 [4,8] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.17 [4,7,5,8] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.18 [3,9] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.24 [4,9] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.27 [4,6] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.33 [5,9] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.3 [5,8] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.5 [9,10] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.12 [9,10] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.19 [9,8] 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1356730303 expires 2024-03-12T19:41:54.175068+0000 2024-03-11T19:53:00.293 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/3464148109 expires 2024-03-12T19:41:54.175068+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/865925559 expires 2024-03-12T19:41:19.200454+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/531077471 expires 2024-03-12T19:41:19.200454+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/3464148109 expires 2024-03-12T19:41:54.175068+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1620686143 expires 2024-03-12T19:41:54.175068+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/927061211 expires 2024-03-12T19:41:19.200454+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/927061211 expires 2024-03-12T19:41:19.200454+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2887016959 expires 2024-03-12T19:41:54.175068+0000 2024-03-11T19:53:00.294 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3318828522 expires 2024-03-12T19:41:19.200454+0000 2024-03-11T19:53:00.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 msgr2=0x7fc7380a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:00.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 0x7fc7380a3f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc744005950 tx=0x7fc74400b410 comp rx=0 tx=0).stop 2024-03-11T19:53:00.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 msgr2=0x7fc75419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:00.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc7400095c0 tx=0x7fc74000f690 comp rx=0 tx=0).stop 2024-03-11T19:53:00.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 shutdown_connections 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fc7380a1ac0 0x7fc7380a3f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc754109ba0 0x7fc7541a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc754074650 0x7fc754199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 --2- 172.21.15.107:0/2947454299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc754075c20 0x7fc75419a3a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.286+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 >> 172.21.15.107:0/2947454299 conn(0x7fc7540fd280 msgr2=0x7fc754105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.287+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 shutdown_connections 2024-03-11T19:53:00.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:00.287+0000 7fc759c00700 1 -- 172.21.15.107:0/2947454299 wait complete. 2024-03-11T19:53:00.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:00 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2947454299' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:53:00.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:00 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2947454299' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:53:00.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:00 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2947454299' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:53:01.193 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-11T19:53:01.204 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 -- 172.21.15.107:0/2413420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 msgr2=0x7f94901037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 --2- 172.21.15.107:0/2413420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f94901037e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9480009900 tx=0x7f948002bc40 comp rx=0 tx=0).stop 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 -- 172.21.15.107:0/2413420598 shutdown_connections 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 --2- 172.21.15.107:0/2413420598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9490104f70 0x7f949010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 --2- 172.21.15.107:0/2413420598 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f94901045b0 0x7f9490104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 --2- 172.21.15.107:0/2413420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f94901037e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 -- 172.21.15.107:0/2413420598 >> 172.21.15.107:0/2413420598 conn(0x7f94900fe970 msgr2=0x7f9490100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 -- 172.21.15.107:0/2413420598 shutdown_connections 2024-03-11T19:53:01.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.420+0000 7f9497200640 1 -- 172.21.15.107:0/2413420598 wait complete. 2024-03-11T19:53:01.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 Processor -- start 2024-03-11T19:53:01.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 -- start start 2024-03-11T19:53:01.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94901045b0 0x7f94901999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9490104f70 0x7f94901a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94901a0f70 con 0x7f94901033c0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94901a10e0 con 0x7f94901045b0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f9497200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94901a12a0 con 0x7f9490104f70 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.421+0000 7f948fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f9487400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94901045b0 0x7f94901999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f9495200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9490104f70 0x7f94901a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41662/0 (socket says 172.21.15.107:41662) 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 -- 172.21.15.107:0/546926536 learned_addr learned my addr 172.21.15.107:0/546926536 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 -- 172.21.15.107:0/546926536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9490104f70 msgr2=0x7f94901a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9490104f70 0x7f94901a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 -- 172.21.15.107:0/546926536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94901045b0 msgr2=0x7f94901999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94901045b0 0x7f94901999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 -- 172.21.15.107:0/546926536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009560 con 0x7f94901033c0 2024-03-11T19:53:01.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.422+0000 7f948fe00640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9480005ab0 tx=0x7f9480005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.423+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800043d0 con 0x7f94901033c0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.423+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94800052d0 con 0x7f94901033c0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.423+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94901a1520 con 0x7f94901033c0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.423+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480003d40 con 0x7f94901033c0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.423+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94901a1a30 con 0x7f94901033c0 2024-03-11T19:53:01.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.424+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949010a6e0 con 0x7f94901033c0 2024-03-11T19:53:01.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.426+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9480005440 con 0x7f94901033c0 2024-03-11T19:53:01.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.428+0000 7f948d600640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 0x7f94740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.428+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f94800ef940 con 0x7f94901033c0 2024-03-11T19:53:01.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.428+0000 7f9487400640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 0x7f94740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.429+0000 7f9487400640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 0x7f94740a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f949019a990 tx=0x7f9478005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:01.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.433+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94800b8740 con 0x7f94901033c0 2024-03-11T19:53:01.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.632+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f949002cdd0 con 0x7f94901033c0 2024-03-11T19:53:01.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.637+0000 7f948d600640 1 -- 172.21.15.107:0/546926536 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7f94800bbe00 con 0x7f94901033c0 2024-03-11T19:53:01.642 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:53:01.642 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.639+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 msgr2=0x7f94740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 0x7f94740a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f949019a990 tx=0x7f9478005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 msgr2=0x7f9490199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9480005ab0 tx=0x7f9480005e50 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 shutdown_connections 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f94740a1a30 0x7f94740a3ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94901045b0 0x7f94901999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9490104f70 0x7f94901a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 --2- 172.21.15.107:0/546926536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94901033c0 0x7f9490199480 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 >> 172.21.15.107:0/546926536 conn(0x7f94900fe970 msgr2=0x7f94901078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 shutdown_connections 2024-03-11T19:53:01.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.640+0000 7f9497200640 1 -- 172.21.15.107:0/546926536 wait complete. 2024-03-11T19:53:01.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:53:01.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:53:01.746 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:01 smithi107 ceph-mon[31712]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:01.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:01 smithi122 ceph-mon[36397]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:01.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:01 smithi145 ceph-mon[36730]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 -- 172.21.15.107:0/675977049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 msgr2=0x7efe6c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 --2- 172.21.15.107:0/675977049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c104a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 -- 172.21.15.107:0/675977049 shutdown_connections 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 --2- 172.21.15.107:0/675977049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c104f70 0x7efe6c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 --2- 172.21.15.107:0/675977049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c104a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 --2- 172.21.15.107:0/675977049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c1033c0 0x7efe6c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 -- 172.21.15.107:0/675977049 >> 172.21.15.107:0/675977049 conn(0x7efe6c0fe930 msgr2=0x7efe6c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:01.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 -- 172.21.15.107:0/675977049 shutdown_connections 2024-03-11T19:53:01.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.877+0000 7efe73a00640 1 -- 172.21.15.107:0/675977049 wait complete. 2024-03-11T19:53:01.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.878+0000 7efe73a00640 1 Processor -- start 2024-03-11T19:53:01.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 -- start start 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c1033c0 0x7efe6c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c104f70 0x7efe6c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c1a0f10 con 0x7efe6c1045b0 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c1a1080 con 0x7efe6c104f70 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe73a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c1a1240 con 0x7efe6c1033c0 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe6be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe71000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c1033c0 0x7efe6c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe6be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41674/0 (socket says 172.21.15.107:41674) 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe6be00640 1 -- 172.21.15.107:0/51081035 learned_addr learned my addr 172.21.15.107:0/51081035 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.879+0000 7efe71a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c104f70 0x7efe6c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 -- 172.21.15.107:0/51081035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c1033c0 msgr2=0x7efe6c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c1033c0 0x7efe6c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 -- 172.21.15.107:0/51081035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c104f70 msgr2=0x7efe6c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c104f70 0x7efe6c1a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 -- 172.21.15.107:0/51081035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54009560 con 0x7efe6c1045b0 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe6be00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe540043d0 con 0x7efe6c1045b0 2024-03-11T19:53:01.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c1a14c0 con 0x7efe6c1045b0 2024-03-11T19:53:01.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe540052d0 con 0x7efe6c1045b0 2024-03-11T19:53:01.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54003cc0 con 0x7efe6c1045b0 2024-03-11T19:53:01.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.880+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c1a19d0 con 0x7efe6c1045b0 2024-03-11T19:53:01.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.882+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c10a680 con 0x7efe6c1045b0 2024-03-11T19:53:01.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.883+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7efe54005440 con 0x7efe6c1045b0 2024-03-11T19:53:01.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.885+0000 7efe69600640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 0x7efe500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:01.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.885+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efe540efa30 con 0x7efe6c1045b0 2024-03-11T19:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.885+0000 7efe71000640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 0x7efe500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:01.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.885+0000 7efe71000640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 0x7efe500a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe6c104410 tx=0x7efe5c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:01.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:01.889+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe540b8830 con 0x7efe6c1045b0 2024-03-11T19:53:02.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.085+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe6c02ce40 con 0x7efe6c1045b0 2024-03-11T19:53:02.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.086+0000 7efe69600640 1 -- 172.21.15.107:0/51081035 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe540bbef0 con 0x7efe6c1045b0 2024-03-11T19:53:02.091 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:53:02.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.088+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 msgr2=0x7efe500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.088+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 0x7efe500a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe6c104410 tx=0x7efe5c009500 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.088+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 msgr2=0x7efe6c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.088+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54005e80 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 shutdown_connections 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efe500a1a30 0x7efe500a3ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe6c104f70 0x7efe6c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe6c1045b0 0x7efe6c199960 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 --2- 172.21.15.107:0/51081035 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe6c1033c0 0x7efe6c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 >> 172.21.15.107:0/51081035 conn(0x7efe6c0fe930 msgr2=0x7efe6c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 shutdown_connections 2024-03-11T19:53:02.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.089+0000 7efe73a00640 1 -- 172.21.15.107:0/51081035 wait complete. 2024-03-11T19:53:02.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:53:02.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.320+0000 7f7b3d200640 1 -- 172.21.15.107:0/1896870733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104e50 msgr2=0x7f7b3810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.320+0000 7f7b3d200640 1 --2- 172.21.15.107:0/1896870733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104e50 0x7f7b3810b6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009be0 tx=0x7f7b3002bef0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 -- 172.21.15.107:0/1896870733 shutdown_connections 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 --2- 172.21.15.107:0/1896870733 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104e50 0x7f7b3810b6e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 --2- 172.21.15.107:0/1896870733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38104490 0x7f7b38104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 --2- 172.21.15.107:0/1896870733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b381032a0 0x7f7b381036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 -- 172.21.15.107:0/1896870733 >> 172.21.15.107:0/1896870733 conn(0x7f7b380fe830 msgr2=0x7f7b38100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 -- 172.21.15.107:0/1896870733 shutdown_connections 2024-03-11T19:53:02.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.321+0000 7f7b3d200640 1 -- 172.21.15.107:0/1896870733 wait complete. 2024-03-11T19:53:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 Processor -- start 2024-03-11T19:53:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 -- start start 2024-03-11T19:53:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104490 0x7f7b381997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 0x7f7b381a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a0d90 con 0x7f7b381032a0 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a0f00 con 0x7f7b38104e50 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.322+0000 7f7b3d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a10c0 con 0x7f7b38104490 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41682/0 (socket says 172.21.15.107:41682) 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 -- 172.21.15.107:0/3468590954 learned_addr learned my addr 172.21.15.107:0/3468590954 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b2ea00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104490 0x7f7b381997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37e00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 0x7f7b381a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 -- 172.21.15.107:0/3468590954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104490 msgr2=0x7f7b381997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104490 0x7f7b381997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 -- 172.21.15.107:0/3468590954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 msgr2=0x7f7b381a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 0x7f7b381a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 -- 172.21.15.107:0/3468590954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30009840 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37e00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 0x7f7b381a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b37400640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d770 tx=0x7f7b2800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.323+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004220 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.324+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b28004380 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.324+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28003df0 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.324+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b381a13a0 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.324+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b381a18b0 con 0x7f7b381032a0 2024-03-11T19:53:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.325+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3810a560 con 0x7f7b381032a0 2024-03-11T19:53:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.327+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7b28009820 con 0x7f7b381032a0 2024-03-11T19:53:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.329+0000 7f7b34c00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 0x7f7b1c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.329+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b280cd5f0 con 0x7f7b381032a0 2024-03-11T19:53:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.329+0000 7f7b2ea00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 0x7f7b1c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.330+0000 7f7b2ea00640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 0x7f7b1c0a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b3819a7b0 tx=0x7f7b20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.332+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b280963f0 con 0x7f7b381032a0 2024-03-11T19:53:02.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.529+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b3810ec80 con 0x7f7b381032a0 2024-03-11T19:53:02.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.530+0000 7f7b34c00640 1 -- 172.21.15.107:0/3468590954 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b280200a0 con 0x7f7b381032a0 2024-03-11T19:53:02.535 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 msgr2=0x7f7b1c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 0x7f7b1c0a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b3819a7b0 tx=0x7f7b20009260 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 msgr2=0x7f7b381992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d770 tx=0x7f7b2800db70 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 shutdown_connections 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7b1c0a1a30 0x7f7b1c0a3ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b38104e50 0x7f7b381a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7b38104490 0x7f7b381997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 --2- 172.21.15.107:0/3468590954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b381032a0 0x7f7b381992a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 >> 172.21.15.107:0/3468590954 conn(0x7f7b380fe830 msgr2=0x7f7b38107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 shutdown_connections 2024-03-11T19:53:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.533+0000 7f7b3d200640 1 -- 172.21.15.107:0/3468590954 wait complete. 2024-03-11T19:53:02.556 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:53:02.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 -- 172.21.15.107:0/1194458402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e41033a0 msgr2=0x7f61e41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 --2- 172.21.15.107:0/1194458402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e41033a0 0x7f61e41037c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009900 tx=0x7f61d402bc40 comp rx=0 tx=0).stop 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 -- 172.21.15.107:0/1194458402 shutdown_connections 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 --2- 172.21.15.107:0/1194458402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e4104f50 0x7f61e410b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 --2- 172.21.15.107:0/1194458402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104590 0x7f61e4104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 --2- 172.21.15.107:0/1194458402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e41033a0 0x7f61e41037c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 -- 172.21.15.107:0/1194458402 >> 172.21.15.107:0/1194458402 conn(0x7f61e40fe930 msgr2=0x7f61e4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 -- 172.21.15.107:0/1194458402 shutdown_connections 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.756+0000 7f61e9800640 1 -- 172.21.15.107:0/1194458402 wait complete. 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 Processor -- start 2024-03-11T19:53:02.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 -- start start 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 0x7f61e4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104f50 0x7f61e41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a0f40 con 0x7f61e4104590 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a10b0 con 0x7f61e41033a0 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.757+0000 7f61e9800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41a1270 con 0x7f61e4104f50 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 0x7f61e4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:54656/0 (socket says 172.21.15.107:54656) 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2a00640 1 -- 172.21.15.107:0/3060075197 learned_addr learned my addr 172.21.15.107:0/3060075197 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e3400640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104f50 0x7f61e41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 -- 172.21.15.107:0/3060075197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104f50 msgr2=0x7f61e41a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104f50 0x7f61e41a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 -- 172.21.15.107:0/3060075197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 msgr2=0x7f61e4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 -- 172.21.15.107:0/3060075197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4009560 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2a00640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.758+0000 7f61e2000640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 0x7f61e4199990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9f0 tx=0x7f61cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.759+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00ccc0 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.759+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61cc00ce20 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.759+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e41a1550 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.759+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004a00 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.759+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e41a1a60 con 0x7f61e4104590 2024-03-11T19:53:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.760+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e410a6c0 con 0x7f61e4104590 2024-03-11T19:53:02.769 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/546926536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:02.769 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:02 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/51081035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.762+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f61cc0050b0 con 0x7f61e4104590 2024-03-11T19:53:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.763+0000 7f61db400640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 0x7f61c80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.763+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61cc014070 con 0x7f61e4104590 2024-03-11T19:53:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.763+0000 7f61e2a00640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 0x7f61c80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:02.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.764+0000 7f61e2a00640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 0x7f61c80a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61d4006010 tx=0x7f61d402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:02.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.767+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61cc096510 con 0x7f61e4104590 2024-03-11T19:53:02.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/546926536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:02.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:02 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/51081035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/546926536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:02.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:02 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/51081035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:02.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.968+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61e402cdd0 con 0x7f61e4104590 2024-03-11T19:53:02.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.970+0000 7f61db400640 1 -- 172.21.15.107:0/3060075197 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61cc099bd0 con 0x7f61e4104590 2024-03-11T19:53:02.974 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:53:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.972+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 msgr2=0x7f61c80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.972+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 0x7f61c80a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61d4006010 tx=0x7f61d402c7f0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.972+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 msgr2=0x7f61e4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.972+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 0x7f61e4199990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e9f0 tx=0x7f61cc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.972+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 shutdown_connections 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f61c80a1ab0 0x7f61c80a3f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61e4104f50 0x7f61e41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4104590 0x7f61e4199990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 --2- 172.21.15.107:0/3060075197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61e41033a0 0x7f61e4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 >> 172.21.15.107:0/3060075197 conn(0x7f61e40fe930 msgr2=0x7f61e4107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 shutdown_connections 2024-03-11T19:53:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:02.973+0000 7f61e9800640 1 -- 172.21.15.107:0/3060075197 wait complete. 2024-03-11T19:53:02.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:53:03.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 -- 172.21.15.107:0/2771587443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54108990 msgr2=0x7f4e54108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/2771587443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54108990 0x7f4e54108e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009900 tx=0x7f4e3c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:03.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 -- 172.21.15.107:0/2771587443 shutdown_connections 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/2771587443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54108990 0x7f4e54108e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/2771587443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e54109350 0x7f4e5410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/2771587443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e541077a0 0x7f4e54107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 -- 172.21.15.107:0/2771587443 >> 172.21.15.107:0/2771587443 conn(0x7f4e54076740 msgr2=0x7f4e54078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 -- 172.21.15.107:0/2771587443 shutdown_connections 2024-03-11T19:53:03.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.193+0000 7f4e5aa00640 1 -- 172.21.15.107:0/2771587443 wait complete. 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.194+0000 7f4e5aa00640 1 Processor -- start 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.194+0000 7f4e5aa00640 1 -- start start 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.194+0000 7f4e5aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e541077a0 0x7f4e5419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.194+0000 7f4e5aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e5aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54109350 0x7f4e541a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e5aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a5330 con 0x7f4e541077a0 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e5aa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a54a0 con 0x7f4e54109350 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e5aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a5660 con 0x7f4e54108990 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40428/0 (socket says 172.21.15.107:40428) 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 -- 172.21.15.107:0/3420742443 learned_addr learned my addr 172.21.15.107:0/3420742443 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e53e00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e541077a0 0x7f4e5419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e58a00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54109350 0x7f4e541a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 -- 172.21.15.107:0/3420742443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54109350 msgr2=0x7f4e541a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54109350 0x7f4e541a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 -- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e541077a0 msgr2=0x7f4e5419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e541077a0 0x7f4e5419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.195+0000 7f4e4b400640 1 -- 172.21.15.107:0/3420742443 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009560 con 0x7f4e54108990 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e4b400640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006010 tx=0x7f4e3c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:03.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0043d0 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e541a58e0 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e3c0052d0 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c003cc0 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.196+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e541a5e40 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.197+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5410eac0 con 0x7f4e54108990 2024-03-11T19:53:03.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.199+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4e3c005440 con 0x7f4e54108990 2024-03-11T19:53:03.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.200+0000 7f4e51600640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 0x7f4e380a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.200+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4e3c0eff60 con 0x7f4e54108990 2024-03-11T19:53:03.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.201+0000 7f4e53e00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 0x7f4e380a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.201+0000 7f4e53e00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 0x7f4e380a3f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e541087f0 tx=0x7f4e44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:03.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.203+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e3c0b8d60 con 0x7f4e54108990 2024-03-11T19:53:03.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.388+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4e541133f0 con 0x7f4e54108990 2024-03-11T19:53:03.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.389+0000 7f4e51600640 1 -- 172.21.15.107:0/3420742443 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4e3c043020 con 0x7f4e54108990 2024-03-11T19:53:03.394 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 msgr2=0x7f4e380a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 0x7f4e380a3f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e541087f0 tx=0x7f4e44005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 msgr2=0x7f4e5419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c006010 tx=0x7f4e3c005e80 comp rx=0 tx=0).stop 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 shutdown_connections 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4e380a1ab0 0x7f4e380a3f70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e54109350 0x7f4e541a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4e54108990 0x7f4e5419dd80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 --2- 172.21.15.107:0/3420742443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e541077a0 0x7f4e5419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 >> 172.21.15.107:0/3420742443 conn(0x7f4e54076740 msgr2=0x7f4e5410bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 shutdown_connections 2024-03-11T19:53:03.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.391+0000 7f4e5aa00640 1 -- 172.21.15.107:0/3420742443 wait complete. 2024-03-11T19:53:03.414 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:53:03.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 -- 172.21.15.107:0/3741379934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c4109370 msgr2=0x7f83c410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 --2- 172.21.15.107:0/3741379934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c4109370 0x7f83c410fc00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ca0 tx=0x7f83bc02c100 comp rx=0 tx=0).stop 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 -- 172.21.15.107:0/3741379934 shutdown_connections 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 --2- 172.21.15.107:0/3741379934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c41089b0 0x7f83c4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 --2- 172.21.15.107:0/3741379934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c4109370 0x7f83c410fc00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 --2- 172.21.15.107:0/3741379934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41077c0 0x7f83c4107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 -- 172.21.15.107:0/3741379934 >> 172.21.15.107:0/3741379934 conn(0x7f83c4076790 msgr2=0x7f83c4078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 -- 172.21.15.107:0/3741379934 shutdown_connections 2024-03-11T19:53:03.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.611+0000 7f83c9a00640 1 -- 172.21.15.107:0/3741379934 wait complete. 2024-03-11T19:53:03.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 Processor -- start 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 -- start start 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 0x7f83c419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c4109370 0x7f83c41a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41a52f0 con 0x7f83c41089b0 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41a5460 con 0x7f83c4109370 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.612+0000 7f83c9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41a5620 con 0x7f83c41077c0 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83c2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 0x7f83c419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83c2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 0x7f83c419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:40446/0 (socket says 172.21.15.107:40446) 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41740/0 (socket says 172.21.15.107:41740) 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83c2a00640 1 -- 172.21.15.107:0/466618537 learned_addr learned my addr 172.21.15.107:0/466618537 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83c3400640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c4109370 0x7f83c41a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 -- 172.21.15.107:0/466618537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 msgr2=0x7f83c419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 0x7f83c419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 -- 172.21.15.107:0/466618537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c4109370 msgr2=0x7f83c41a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c4109370 0x7f83c41a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 -- 172.21.15.107:0/466618537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc009900 con 0x7f83c41089b0 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.613+0000 7f83ba000640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e9f0 tx=0x7f83ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.614+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00ccc0 con 0x7f83c41089b0 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.614+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83ac00ce20 con 0x7f83c41089b0 2024-03-11T19:53:03.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.614+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac004a00 con 0x7f83c41089b0 2024-03-11T19:53:03.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.614+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c41a5900 con 0x7f83c41089b0 2024-03-11T19:53:03.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.614+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c41a5dc0 con 0x7f83c41089b0 2024-03-11T19:53:03.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.615+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c410eae0 con 0x7f83c41089b0 2024-03-11T19:53:03.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.617+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f83ac0050b0 con 0x7f83c41089b0 2024-03-11T19:53:03.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.619+0000 7f83bbe00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 0x7f83a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:03.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.619+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f83ac014070 con 0x7f83c41089b0 2024-03-11T19:53:03.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.619+0000 7f83c2a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 0x7f83a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:03.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.620+0000 7f83c2a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 0x7f83a80a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83b40058c0 tx=0x7f83b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:03.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.623+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83ac096590 con 0x7f83c41089b0 2024-03-11T19:53:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:03 smithi122 ceph-mon[36397]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:03 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3468590954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:03 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3060075197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:03 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3420742443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:03 smithi145 ceph-mon[36730]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:03 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3468590954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:03 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3060075197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:03 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3420742443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:03 smithi107 ceph-mon[31712]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:03.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3468590954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3060075197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:03 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3420742443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:03.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.859+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83c41131c0 con 0x7f83c41089b0 2024-03-11T19:53:03.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.861+0000 7f83bbe00640 1 -- 172.21.15.107:0/466618537 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f83ac094e60 con 0x7f83c41089b0 2024-03-11T19:53:03.866 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:53:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:53:03.867 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 msgr2=0x7f83a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 0x7f83a80a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83b40058c0 tx=0x7f83b4005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 msgr2=0x7f83c419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e9f0 tx=0x7f83ac00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 shutdown_connections 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f83a80a1a30 0x7f83a80a3ef0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c4109370 0x7f83c41a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c41089b0 0x7f83c419dd40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 --2- 172.21.15.107:0/466618537 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f83c41077c0 0x7f83c419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 >> 172.21.15.107:0/466618537 conn(0x7f83c4076790 msgr2=0x7f83c410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:03.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 shutdown_connections 2024-03-11T19:53:03.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:03.864+0000 7f83c9a00640 1 -- 172.21.15.107:0/466618537 wait complete. 2024-03-11T19:53:03.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 -- 172.21.15.107:0/2500962890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 msgr2=0x7f6e58104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 --2- 172.21.15.107:0/2500962890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58104840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802bc50 comp rx=0 tx=0).stop 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 -- 172.21.15.107:0/2500962890 shutdown_connections 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 --2- 172.21.15.107:0/2500962890 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e58104d80 0x7f6e5810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 --2- 172.21.15.107:0/2500962890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58104840 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 --2- 172.21.15.107:0/2500962890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e581031d0 0x7f6e581035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 -- 172.21.15.107:0/2500962890 >> 172.21.15.107:0/2500962890 conn(0x7f6e580fe760 msgr2=0x7f6e58100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:04.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 -- 172.21.15.107:0/2500962890 shutdown_connections 2024-03-11T19:53:04.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.096+0000 7f6e5f200640 1 -- 172.21.15.107:0/2500962890 wait complete. 2024-03-11T19:53:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.097+0000 7f6e5f200640 1 Processor -- start 2024-03-11T19:53:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 -- start start 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e581031d0 0x7f6e58199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e58104d80 0x7f6e581a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a0cf0 con 0x7f6e581043c0 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a0e60 con 0x7f6e58104d80 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581a1020 con 0x7f6e581031d0 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41762/0 (socket says 172.21.15.107:41762) 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5d400640 1 -- 172.21.15.107:0/3109518710 learned_addr learned my addr 172.21.15.107:0/3109518710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.098+0000 7f6e5de00640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e581031d0 0x7f6e58199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5e800640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e58104d80 0x7f6e581a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 -- 172.21.15.107:0/3109518710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e581031d0 msgr2=0x7f6e58199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e581031d0 0x7f6e58199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 -- 172.21.15.107:0/3109518710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e58104d80 msgr2=0x7f6e581a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e58104d80 0x7f6e581a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 -- 172.21.15.107:0/3109518710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009560 con 0x7f6e581043c0 2024-03-11T19:53:04.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5d400640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e4802c6c0 tx=0x7f6e480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480044c0 con 0x7f6e581043c0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e581a12a0 con 0x7f6e581043c0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.099+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e480052d0 con 0x7f6e581043c0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.100+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4803e450 con 0x7f6e581043c0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.100+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581a1800 con 0x7f6e581043c0 2024-03-11T19:53:04.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.101+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5810a490 con 0x7f6e581043c0 2024-03-11T19:53:04.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.103+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f6e48005440 con 0x7f6e581043c0 2024-03-11T19:53:04.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.104+0000 7f6e52a00640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 0x7f6e400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.104+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6e480f0040 con 0x7f6e581043c0 2024-03-11T19:53:04.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.104+0000 7f6e5de00640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 0x7f6e400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.105+0000 7f6e5de00640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 0x7f6e400a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e58104220 tx=0x7f6e4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:04.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.108+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e480b8e80 con 0x7f6e581043c0 2024-03-11T19:53:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.299+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e58109f00 con 0x7f6e581043c0 2024-03-11T19:53:04.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.303+0000 7f6e52a00640 1 -- 172.21.15.107:0/3109518710 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7f6e48043020 con 0x7f6e581043c0 2024-03-11T19:53:04.308 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:53:04.309 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:53:04.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 msgr2=0x7f6e400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 0x7f6e400a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e58104220 tx=0x7f6e4c005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 msgr2=0x7f6e58199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e4802c6c0 tx=0x7f6e480040c0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 shutdown_connections 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f6e400a1a30 0x7f6e400a3ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e58104d80 0x7f6e581a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e581043c0 0x7f6e58199740 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 --2- 172.21.15.107:0/3109518710 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e581031d0 0x7f6e58199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 >> 172.21.15.107:0/3109518710 conn(0x7f6e580fe760 msgr2=0x7f6e58107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 shutdown_connections 2024-03-11T19:53:04.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.306+0000 7f6e5f200640 1 -- 172.21.15.107:0/3109518710 wait complete. 2024-03-11T19:53:04.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T19:53:04.331 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 -- 172.21.15.107:0/2769162305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4101190 msgr2=0x7f96e41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 --2- 172.21.15.107:0/2769162305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4101190 0x7f96e41015b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009900 tx=0x7f96d802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 -- 172.21.15.107:0/2769162305 shutdown_connections 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 --2- 172.21.15.107:0/2769162305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102380 0x7f96e4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 --2- 172.21.15.107:0/2769162305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4102d40 0x7f96e41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 --2- 172.21.15.107:0/2769162305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4101190 0x7f96e41015b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 -- 172.21.15.107:0/2769162305 >> 172.21.15.107:0/2769162305 conn(0x7f96e40fc720 msgr2=0x7f96e40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 -- 172.21.15.107:0/2769162305 shutdown_connections 2024-03-11T19:53:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.543+0000 7f96ec800640 1 -- 172.21.15.107:0/2769162305 wait complete. 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.544+0000 7f96ec800640 1 Processor -- start 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.544+0000 7f96ec800640 1 -- start start 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4101190 0x7f96e4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102d40 0x7f96e419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e419ed50 con 0x7f96e4102380 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e419eec0 con 0x7f96e4102d40 2024-03-11T19:53:04.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ec800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e419f080 con 0x7f96e4101190 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41790/0 (socket says 172.21.15.107:41790) 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4101190 0x7f96e4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 -- 172.21.15.107:0/217736298 learned_addr learned my addr 172.21.15.107:0/217736298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96ea800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102d40 0x7f96e419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 -- 172.21.15.107:0/217736298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4101190 msgr2=0x7f96e4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4101190 0x7f96e4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 -- 172.21.15.107:0/217736298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102d40 msgr2=0x7f96e419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102d40 0x7f96e419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.545+0000 7f96e9400640 1 -- 172.21.15.107:0/217736298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8009560 con 0x7f96e4102380 2024-03-11T19:53:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96e9400640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96d400e9f0 tx=0x7f96d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4009700 con 0x7f96e4102380 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d400cd80 con 0x7f96e4102380 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4018920 con 0x7f96e4102380 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e419f360 con 0x7f96e4102380 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.546+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e419f870 con 0x7f96e4102380 2024-03-11T19:53:04.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.547+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e4108520 con 0x7f96e4102380 2024-03-11T19:53:04.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.549+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f96d4018a80 con 0x7f96e4102380 2024-03-11T19:53:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.550+0000 7f96dea00640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 0x7f96cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.550+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f96d4014070 con 0x7f96e4102380 2024-03-11T19:53:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.551+0000 7f96e9e00640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 0x7f96cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.551+0000 7f96e9e00640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 0x7f96cc0a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005ab0 tx=0x7f96d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.555+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96d4052d20 con 0x7f96e4102380 2024-03-11T19:53:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.754+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96e402cda0 con 0x7f96e4102380 2024-03-11T19:53:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.755+0000 7f96dea00640 1 -- 172.21.15.107:0/217736298 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96d4096a50 con 0x7f96e4102380 2024-03-11T19:53:04.760 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T19:53:04.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:04 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/466618537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:04 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3109518710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 msgr2=0x7f96cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 0x7f96cc0a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005ab0 tx=0x7f96d8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 msgr2=0x7f96e41977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96d400e9f0 tx=0x7f96d400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 shutdown_connections 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f96cc0a1a30 0x7f96cc0a3ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e4102d40 0x7f96e419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96e4102380 0x7f96e41977a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 --2- 172.21.15.107:0/217736298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f96e4101190 0x7f96e4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.758+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 >> 172.21.15.107:0/217736298 conn(0x7f96e40fc720 msgr2=0x7f96e41056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:04.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.759+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 shutdown_connections 2024-03-11T19:53:04.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.759+0000 7f96ec800640 1 -- 172.21.15.107:0/217736298 wait complete. 2024-03-11T19:53:04.779 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:04 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/466618537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.780 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:04 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3109518710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:53:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:04 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/466618537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:04 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3109518710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 -- 172.21.15.107:0/3126751049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 msgr2=0x7f25f41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 --2- 172.21.15.107:0/3126751049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f41037e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009900 tx=0x7f25e402bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 -- 172.21.15.107:0/3126751049 shutdown_connections 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 --2- 172.21.15.107:0/3126751049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 0x7f25f410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 --2- 172.21.15.107:0/3126751049 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 --2- 172.21.15.107:0/3126751049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f41037e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:04.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 -- 172.21.15.107:0/3126751049 >> 172.21.15.107:0/3126751049 conn(0x7f25f40fe930 msgr2=0x7f25f4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:04.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 -- 172.21.15.107:0/3126751049 shutdown_connections 2024-03-11T19:53:04.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.993+0000 7f25f9a00640 1 -- 172.21.15.107:0/3126751049 wait complete. 2024-03-11T19:53:05.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.994+0000 7f25f9a00640 1 Processor -- start 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.994+0000 7f25f9a00640 1 -- start start 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.994+0000 7f25f9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.994+0000 7f25f9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 0x7f25f41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f9a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a0f70 con 0x7f25f41033c0 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a10e0 con 0x7f25f4104f70 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a12a0 con 0x7f25f41045b0 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55486/0 (socket says 172.21.15.107:55486) 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 -- 172.21.15.107:0/1580632275 learned_addr learned my addr 172.21.15.107:0/1580632275 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f3400640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 0x7f25f41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 -- 172.21.15.107:0/1580632275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 msgr2=0x7f25f41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 0x7f25f41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 -- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 msgr2=0x7f25f4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.995+0000 7f25f2000640 1 -- 172.21.15.107:0/1580632275 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4009560 con 0x7f25f41045b0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25f2000640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9f0 tx=0x7f25dc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc009720 con 0x7f25f41045b0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f41a1580 con 0x7f25f41045b0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25dc00cd80 con 0x7f25f41045b0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc010690 con 0x7f25f41045b0 2024-03-11T19:53:05.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.996+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f41a1a40 con 0x7f25f41045b0 2024-03-11T19:53:05.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.997+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f410a6e0 con 0x7f25f41045b0 2024-03-11T19:53:05.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:04.999+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f25dc0046c0 con 0x7f25f41045b0 2024-03-11T19:53:05.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.000+0000 7f25eb400640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 0x7f25d80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.000+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f25dc014070 con 0x7f25f41045b0 2024-03-11T19:53:05.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.001+0000 7f25f2a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 0x7f25d80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.001+0000 7f25f2a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 0x7f25d80a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25e4005ab0 tx=0x7f25e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.004+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25dc096980 con 0x7f25f41045b0 2024-03-11T19:53:05.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.202+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25f402cdd0 con 0x7f25f41045b0 2024-03-11T19:53:05.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.203+0000 7f25eb400640 1 -- 172.21.15.107:0/1580632275 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25dc09a040 con 0x7f25f41045b0 2024-03-11T19:53:05.208 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 msgr2=0x7f25d80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 0x7f25d80a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25e4005ab0 tx=0x7f25e4005fd0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 msgr2=0x7f25f41999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9f0 tx=0x7f25dc00edf0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 shutdown_connections 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f25d80a1a30 0x7f25d80a3ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f4104f70 0x7f25f41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25f41045b0 0x7f25f41999c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 --2- 172.21.15.107:0/1580632275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25f41033c0 0x7f25f4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 >> 172.21.15.107:0/1580632275 conn(0x7f25f40fe930 msgr2=0x7f25f41078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 shutdown_connections 2024-03-11T19:53:05.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.206+0000 7f25f9a00640 1 -- 172.21.15.107:0/1580632275 wait complete. 2024-03-11T19:53:05.229 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 -- 172.21.15.107:0/4051672050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308075b40 msgr2=0x7f7308075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 --2- 172.21.15.107:0/4051672050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308075b40 0x7f7308075f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009900 tx=0x7f72f802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 -- 172.21.15.107:0/4051672050 shutdown_connections 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 --2- 172.21.15.107:0/4051672050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308076d80 0x7f73080771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 --2- 172.21.15.107:0/4051672050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308077720 0x7f7308077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 --2- 172.21.15.107:0/4051672050 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308075b40 0x7f7308075f60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 -- 172.21.15.107:0/4051672050 >> 172.21.15.107:0/4051672050 conn(0x7f73080fe930 msgr2=0x7f7308100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 -- 172.21.15.107:0/4051672050 shutdown_connections 2024-03-11T19:53:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.441+0000 7f730fe00640 1 -- 172.21.15.107:0/4051672050 wait complete. 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 Processor -- start 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 -- start start 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 0x7f730819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 0x7f730819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 0x7f73081a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73081a5310 con 0x7f7308076d80 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73081a5480 con 0x7f7308075b40 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.442+0000 7f730fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73081a5640 con 0x7f7308077720 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 0x7f730819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 0x7f730819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36440/0 (socket says 172.21.15.107:36440) 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730ca00640 1 -- 172.21.15.107:0/3967543504 learned_addr learned my addr 172.21.15.107:0/3967543504 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 0x7f73081a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730d400640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 0x7f730819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 -- 172.21.15.107:0/3967543504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 msgr2=0x7f730819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 0x7f730819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 -- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 msgr2=0x7f730819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 0x7f730819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 -- 172.21.15.107:0/3967543504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8009560 con 0x7f7308077720 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730d400640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 0x7f730819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.443+0000 7f730de00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 0x7f73081a4dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7300009fd0 tx=0x7f730000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.444+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000ce20 con 0x7f7308077720 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.444+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7300005020 con 0x7f7308077720 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.444+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73081a5920 con 0x7f7308077720 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.444+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300010690 con 0x7f7308077720 2024-03-11T19:53:05.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.444+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73081a5e30 con 0x7f7308077720 2024-03-11T19:53:05.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.445+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730810fd30 con 0x7f7308077720 2024-03-11T19:53:05.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.447+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f7300005500 con 0x7f7308077720 2024-03-11T19:53:05.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.448+0000 7f7306000640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 0x7f72ec0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.448+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7300014070 con 0x7f7308077720 2024-03-11T19:53:05.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.448+0000 7f730d400640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 0x7f72ec0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.449+0000 7f730d400640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 0x7f72ec0a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009230 tx=0x7f72f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.450+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7300096ca0 con 0x7f7308077720 2024-03-11T19:53:05.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.649+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73081131c0 con 0x7f7308077720 2024-03-11T19:53:05.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.650+0000 7f7306000640 1 -- 172.21.15.107:0/3967543504 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73000ba030 con 0x7f7308077720 2024-03-11T19:53:05.655 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.652+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 msgr2=0x7f72ec0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.652+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 0x7f72ec0a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009230 tx=0x7f72f8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.652+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 msgr2=0x7f73081a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.652+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 0x7f73081a4dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7300009fd0 tx=0x7f730000ef10 comp rx=0 tx=0).stop 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 shutdown_connections 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f72ec0a1ab0 0x7f72ec0a3f70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7308077720 0x7f73081a4dd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7308076d80 0x7f730819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 --2- 172.21.15.107:0/3967543504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7308075b40 0x7f730819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 >> 172.21.15.107:0/3967543504 conn(0x7f73080fe930 msgr2=0x7f7308100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:05.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 shutdown_connections 2024-03-11T19:53:05.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.653+0000 7f730fe00640 1 -- 172.21.15.107:0/3967543504 wait complete. 2024-03-11T19:53:05.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T19:53:05.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:05 smithi145 ceph-mon[36730]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:05.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:05 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/217736298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:05.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:05 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1580632275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:05 smithi107 ceph-mon[31712]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:05 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/217736298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:05.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:05 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1580632275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:05.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 -- 172.21.15.107:0/525638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801045b0 msgr2=0x7fae80104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 --2- 172.21.15.107:0/525638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801045b0 0x7fae80104a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae68009900 tx=0x7fae6802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:05.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 -- 172.21.15.107:0/525638 shutdown_connections 2024-03-11T19:53:05.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 --2- 172.21.15.107:0/525638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801045b0 0x7fae80104a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 --2- 172.21.15.107:0/525638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae80104f70 0x7fae8010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 --2- 172.21.15.107:0/525638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae801033c0 0x7fae801037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 -- 172.21.15.107:0/525638 >> 172.21.15.107:0/525638 conn(0x7fae800fe930 msgr2=0x7fae80100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:05.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 -- 172.21.15.107:0/525638 shutdown_connections 2024-03-11T19:53:05.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.891+0000 7fae85c00640 1 -- 172.21.15.107:0/525638 wait complete. 2024-03-11T19:53:05.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.892+0000 7fae85c00640 1 Processor -- start 2024-03-11T19:53:05.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 -- start start 2024-03-11T19:53:05.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 0x7fae80199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 0x7fae801a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a0f20 con 0x7fae80104f70 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a1090 con 0x7fae801033c0 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae85c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a1250 con 0x7fae801045b0 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44692/0 (socket says 172.21.15.107:44692) 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae7ea00640 1 -- 172.21.15.107:0/2918725158 learned_addr learned my addr 172.21.15.107:0/2918725158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae7f400640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 0x7fae801a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.893+0000 7fae7e000640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 0x7fae80199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 -- 172.21.15.107:0/2918725158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 msgr2=0x7fae80199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 0x7fae80199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 -- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 msgr2=0x7fae801a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 0x7fae801a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:05.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 -- 172.21.15.107:0/2918725158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009560 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7f400640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 0x7fae801a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7e000640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 0x7fae80199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae7ea00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9c0 tx=0x7fae7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78009700 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae7800cd80 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae801a1530 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78010430 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.894+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae801a1a60 con 0x7fae801033c0 2024-03-11T19:53:05.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.895+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8010a6e0 con 0x7fae801033c0 2024-03-11T19:53:05.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.897+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fae78003680 con 0x7fae801033c0 2024-03-11T19:53:05.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.899+0000 7fae77400640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 0x7fae640a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:05.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.899+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae78014070 con 0x7fae801033c0 2024-03-11T19:53:05.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.899+0000 7fae7e000640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 0x7fae640a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:05.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.900+0000 7fae7e000640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 0x7fae640a3f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae68006010 tx=0x7fae6802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:05.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:05.903+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae78096a60 con 0x7fae801033c0 2024-03-11T19:53:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:05 smithi122 ceph-mon[36397]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:05 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/217736298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:05 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1580632275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.100+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fae8010ee20 con 0x7fae801033c0 2024-03-11T19:53:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.101+0000 7fae77400640 1 -- 172.21.15.107:0/2918725158 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fae7809a120 con 0x7fae801033c0 2024-03-11T19:53:06.106 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 msgr2=0x7fae640a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 0x7fae640a3f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fae68006010 tx=0x7fae6802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 msgr2=0x7fae80199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9c0 tx=0x7fae7800ee80 comp rx=0 tx=0).stop 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 shutdown_connections 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fae640a1ab0 0x7fae640a3f70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae80104f70 0x7fae801a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae801045b0 0x7fae80199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 --2- 172.21.15.107:0/2918725158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae801033c0 0x7fae80199430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 >> 172.21.15.107:0/2918725158 conn(0x7fae800fe930 msgr2=0x7fae801078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 shutdown_connections 2024-03-11T19:53:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.104+0000 7fae85c00640 1 -- 172.21.15.107:0/2918725158 wait complete. 2024-03-11T19:53:06.129 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:53:06.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.333+0000 7ff8b6a00640 1 -- 172.21.15.107:0/2330853713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 msgr2=0x7ff8b010b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.333+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/2330853713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b010b800 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009900 tx=0x7ff8a802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 -- 172.21.15.107:0/2330853713 shutdown_connections 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/2330853713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 0x7ff8b0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/2330853713 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b010b800 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/2330853713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 -- 172.21.15.107:0/2330853713 >> 172.21.15.107:0/2330853713 conn(0x7ff8b00fe970 msgr2=0x7ff8b0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 -- 172.21.15.107:0/2330853713 shutdown_connections 2024-03-11T19:53:06.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.334+0000 7ff8b6a00640 1 -- 172.21.15.107:0/2330853713 wait complete. 2024-03-11T19:53:06.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 Processor -- start 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 -- start start 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 0x7ff8b01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b01a0e80 con 0x7ff8b01033c0 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b01a0ff0 con 0x7ff8b01045b0 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b01a11b0 con 0x7ff8b0104f70 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36488/0 (socket says 172.21.15.107:36488) 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8afe00640 1 -- 172.21.15.107:0/1623523263 learned_addr learned my addr 172.21.15.107:0/1623523263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8b4a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.335+0000 7ff8a7400640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 0x7ff8b01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 -- 172.21.15.107:0/1623523263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 msgr2=0x7ff8b01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 -- 172.21.15.107:0/1623523263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 msgr2=0x7ff8b01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 0x7ff8b01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 -- 172.21.15.107:0/1623523263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8009560 con 0x7ff8b01033c0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8b4a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8afe00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000ea80 tx=0x7ff8a000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000ce40 con 0x7ff8b01033c0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0011950 con 0x7ff8b01033c0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b01a1490 con 0x7ff8b01033c0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.336+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001ab40 con 0x7ff8b01033c0 2024-03-11T19:53:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.337+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b01a19a0 con 0x7ff8b01033c0 2024-03-11T19:53:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.338+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b010a610 con 0x7ff8b01033c0 2024-03-11T19:53:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.339+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7ff8a0011ac0 con 0x7ff8b01033c0 2024-03-11T19:53:06.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.342+0000 7ff8ad600640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 0x7ff8940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.342+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff8a00cdd70 con 0x7ff8b01033c0 2024-03-11T19:53:06.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.342+0000 7ff8a7400640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 0x7ff8940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.343+0000 7ff8a7400640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 0x7ff8940a3ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8b019a8a0 tx=0x7ff898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:06.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.346+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8a0096b70 con 0x7ff8b01033c0 2024-03-11T19:53:06.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.588+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8b010ee20 con 0x7ff8b01033c0 2024-03-11T19:53:06.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.590+0000 7ff8ad600640 1 -- 172.21.15.107:0/1623523263 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11358 (secure 0 0 0) 0x7ff8a009a230 con 0x7ff8b01033c0 2024-03-11T19:53:06.595 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:53:06.595 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T19:50:29.435340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:53:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-11T19:53:06.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 msgr2=0x7ff8940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 0x7ff8940a3ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8b019a8a0 tx=0x7ff898005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:06.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 msgr2=0x7ff8b0199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000ea80 tx=0x7ff8a000ef40 comp rx=0 tx=0).stop 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 shutdown_connections 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7ff8940a1a30 0x7ff8940a3ef0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b01045b0 0x7ff8b01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8b0104f70 0x7ff8b01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.592+0000 7ff8b6a00640 1 --2- 172.21.15.107:0/1623523263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b01033c0 0x7ff8b0199390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.593+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 >> 172.21.15.107:0/1623523263 conn(0x7ff8b00fe970 msgr2=0x7ff8b01077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.593+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 shutdown_connections 2024-03-11T19:53:06.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.593+0000 7ff8b6a00640 1 -- 172.21.15.107:0/1623523263 wait complete. 2024-03-11T19:53:06.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:53:06.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:06 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3967543504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:06 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2918725158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:06 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3967543504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:06 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2918725158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:06.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 -- 172.21.15.107:0/3443757378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 msgr2=0x7f4bb4107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 --2- 172.21.15.107:0/3443757378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb4107be0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009860 tx=0x7f4ba402bb70 comp rx=0 tx=0).stop 2024-03-11T19:53:06.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 -- 172.21.15.107:0/3443757378 shutdown_connections 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 --2- 172.21.15.107:0/3443757378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 0x7f4bb410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 --2- 172.21.15.107:0/3443757378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 0x7f4bb4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 --2- 172.21.15.107:0/3443757378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb4107be0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 -- 172.21.15.107:0/3443757378 >> 172.21.15.107:0/3443757378 conn(0x7f4bb4076790 msgr2=0x7f4bb4078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 -- 172.21.15.107:0/3443757378 shutdown_connections 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.825+0000 7f4bbb600640 1 -- 172.21.15.107:0/3443757378 wait complete. 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.826+0000 7f4bbb600640 1 Processor -- start 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 -- start start 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 0x7f4bb419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 0x7f4bb41a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41a5320 con 0x7f4bb41089b0 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41a5490 con 0x7f4bb41077c0 2024-03-11T19:53:06.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bbb600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41a5650 con 0x7f4bb4109370 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44722/0 (socket says 172.21.15.107:44722) 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4babe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 0x7f4bb419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.827+0000 7f4bb8c00640 1 -- 172.21.15.107:0/162194451 learned_addr learned my addr 172.21.15.107:0/162194451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb9600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 0x7f4bb41a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 -- 172.21.15.107:0/162194451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 msgr2=0x7f4bb41a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 0x7f4bb41a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 -- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 msgr2=0x7f4bb419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 0x7f4bb419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:06.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 -- 172.21.15.107:0/162194451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c009560 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb8c00640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402c0b0 tx=0x7f4ba4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba402c510 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4009490 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba402cb50 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.828+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4003850 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.829+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb41a5ab0 con 0x7f4bb41077c0 2024-03-11T19:53:06.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.830+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb410eae0 con 0x7f4bb41077c0 2024-03-11T19:53:06.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.832+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f4ba403b040 con 0x7f4bb41077c0 2024-03-11T19:53:06.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.833+0000 7f4bb2000640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 0x7f4b980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:06.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.833+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ba40ef430 con 0x7f4bb41077c0 2024-03-11T19:53:06.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.833+0000 7f4babe00640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 0x7f4b980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:06.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.834+0000 7f4babe00640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 0x7f4b980a3ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4bb419ed40 tx=0x7f4b9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:06.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:06.836+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ba40b8230 con 0x7f4bb41077c0 2024-03-11T19:53:07.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:06 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3967543504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:53:07.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:06 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2918725158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T19:53:07.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.034+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bb402cdd0 con 0x7f4bb41077c0 2024-03-11T19:53:07.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.037+0000 7f4bb2000640 1 -- 172.21.15.107:0/162194451 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24955 (secure 0 0 0) 0x7f4ba404b090 con 0x7f4bb41077c0 2024-03-11T19:53:07.042 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T19:53:07.042 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":95,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T19:51:56.516846+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 msgr2=0x7f4b980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 0x7f4b980a3ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4bb419ed40 tx=0x7f4b9c009260 comp rx=0 tx=0).stop 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 msgr2=0x7f4bb419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402c0b0 tx=0x7f4ba4005d50 comp rx=0 tx=0).stop 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 shutdown_connections 2024-03-11T19:53:07.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f4b980a1a30 0x7f4b980a3ef0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4bb4109370 0x7f4bb41a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bb41089b0 0x7f4bb419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 --2- 172.21.15.107:0/162194451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bb41077c0 0x7f4bb419d830 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.039+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 >> 172.21.15.107:0/162194451 conn(0x7f4bb4076790 msgr2=0x7f4bb410bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.040+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 shutdown_connections 2024-03-11T19:53:07.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.040+0000 7f4bbb600640 1 -- 172.21.15.107:0/162194451 wait complete. 2024-03-11T19:53:07.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:49:55.178113+0000', 'modified': '2024-03-11T19:50:29.435340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24590, 'mds_1': 24583, 'mds_2': 24547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6835/3941680931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 3941680931}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 3941680931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.145:6837/2305866182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6836', 'nonce': 2305866182}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 2305866182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6837/547863696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 547863696}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 547863696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.145:6839/322644420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6838', 'nonce': 322644420}, {'type': 'v1', 'addr': '172.21.15.145:6839', 'nonce': 322644420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24583': {'gid': 24583, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.107:6841/981878935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 981878935}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 981878935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24590': {'gid': 24590, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6839/1167549353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1167549353}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1167549353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:53:07.066 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:53:07.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:53:07.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 -- 172.21.15.107:0/161024980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c1077a0 msgr2=0x7fde0c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 --2- 172.21.15.107:0/161024980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c1077a0 0x7fde0c107bc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fde00009900 tx=0x7fde0002bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 -- 172.21.15.107:0/161024980 shutdown_connections 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 --2- 172.21.15.107:0/161024980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c108990 0x7fde0c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 --2- 172.21.15.107:0/161024980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c109350 0x7fde0c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 --2- 172.21.15.107:0/161024980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c1077a0 0x7fde0c107bc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.270+0000 7fde14600640 1 -- 172.21.15.107:0/161024980 >> 172.21.15.107:0/161024980 conn(0x7fde0c076740 msgr2=0x7fde0c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.271+0000 7fde14600640 1 -- 172.21.15.107:0/161024980 shutdown_connections 2024-03-11T19:53:07.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.271+0000 7fde14600640 1 -- 172.21.15.107:0/161024980 wait complete. 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.271+0000 7fde14600640 1 Processor -- start 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 -- start start 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 0x7fde0c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c109350 0x7fde0c1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a5310 con 0x7fde0c108990 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a5480 con 0x7fde0c109350 2024-03-11T19:53:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde14600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1a5640 con 0x7fde0c1077a0 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde11200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde11c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 0x7fde0c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde11200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36528/0 (socket says 172.21.15.107:36528) 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde11200640 1 -- 172.21.15.107:0/678334844 learned_addr learned my addr 172.21.15.107:0/678334844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.272+0000 7fde12600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c109350 0x7fde0c1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 -- 172.21.15.107:0/678334844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c109350 msgr2=0x7fde0c1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c109350 0x7fde0c1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 -- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 msgr2=0x7fde0c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 -- 172.21.15.107:0/678334844 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00009560 con 0x7fde0c1077a0 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11200640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde11c00640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 0x7fde0c19d820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde00036040 tx=0x7fde00005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde0a000640 1 -- 172.21.15.107:0/678334844 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000043d0 con 0x7fde0c1077a0 2024-03-11T19:53:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.273+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c1a58c0 con 0x7fde0c1077a0 2024-03-11T19:53:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.274+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c1a5df0 con 0x7fde0c1077a0 2024-03-11T19:53:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.274+0000 7fde0a000640 1 -- 172.21.15.107:0/678334844 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde000052d0 con 0x7fde0c1077a0 2024-03-11T19:53:07.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.274+0000 7fde0a000640 1 -- 172.21.15.107:0/678334844 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00003d00 con 0x7fde0c1077a0 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde0a000640 1 -- 172.21.15.107:0/678334844 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fde00005440 con 0x7fde0c1077a0 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 Processor -- start 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 -- start start 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x564562401570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562368b30 con 0x5645623f9580 2024-03-11T19:53:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562366060 con 0x5645623ff0e0 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.277+0000 7fde173fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562361840 con 0x5645623f9f70 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55594/0 (socket says 172.21.15.107:55594) 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 -- 172.21.15.107:0/73244370 learned_addr learned my addr 172.21.15.107:0/73244370 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde12600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11200640 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x564562401570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 -- 172.21.15.107:0/73244370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 msgr2=0x564562401570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x564562401570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 -- 172.21.15.107:0/73244370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 msgr2=0x5645623f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11c00640 1 -- 172.21.15.107:0/73244370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645623294c0 con 0x5645623f9f70 2024-03-11T19:53:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde11200640 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x564562401570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.278+0000 7fde0a000640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 0x7fddf40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fde11c00640 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde000f5a90 tx=0x7fde000f5b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73c58a878d33710 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fde0a000640 1 -- 172.21.15.107:0/678334844 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fde000ef860 con 0x7fde0c1077a0 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fde12600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 0x7fddf40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fddfea00640 1 -- 172.21.15.107:0/73244370 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0004df00 con 0x5645623f9f70 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fddfea00640 1 -- 172.21.15.107:0/73244370 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde000f56d0 con 0x5645623f9f70 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.279+0000 7fddfea00640 1 -- 172.21.15.107:0/73244370 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000bc190 con 0x5645623f9f70 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- 172.21.15.107:0/73244370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 msgr2=0x5645623fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde12600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 0x7fddf40a3f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5645623fa7d0 tx=0x7fde04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde000f5a90 tx=0x7fde000f5b80 comp rx=0 tx=0).stop 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- 172.21.15.107:0/73244370 shutdown_connections 2024-03-11T19:53:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x564562401570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x5645623fea70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 --2- 172.21.15.107:0/73244370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- 172.21.15.107:0/73244370 >> 172.21.15.107:0/73244370 conn(0x5645623f4af0 msgr2=0x5645623f4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- 172.21.15.107:0/73244370 shutdown_connections 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- 172.21.15.107:0/73244370 wait complete. 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.280+0000 7fde173fa740 1 -- start start 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 Processor -- start 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x564562403160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x56456240a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562306ea0 con 0x5645623ff0e0 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562306b40 con 0x5645623f9f70 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645620a9f20 con 0x5645623f9580 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde173fa740 10 client.0 _ll_register_callbacks cb 0x56456231a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde12600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x56456240a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde12600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x56456240a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36538/0 (socket says 172.21.15.107:36538) 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde12600640 1 -- 172.21.15.107:0/3575435318 learned_addr learned my addr 172.21.15.107:0/3575435318 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde11200640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.281+0000 7fde11c00640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x564562403160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 -- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 msgr2=0x564562403160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x564562403160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 -- 172.21.15.107:0/3575435318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 msgr2=0x56456240a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x56456240a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 -- 172.21.15.107:0/3575435318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c1a5df0 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11c00640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x564562403160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde11200640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624036a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009620 tx=0x7fddf800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7528ba9db5468c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fddf3400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8017070 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fddf80120d0 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.282+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645620616c0 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf8012290 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 10 client.25358 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56456235fec0 con 0x5645623f9f70 2024-03-11T19:53:07.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645620616c0 con 0x5645623f9f70 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fddf8003d20 con 0x5645623f9f70 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 10 client.25358 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 10 client.25358 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 20 client.25358 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456240ea10 0x564562413a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 4 client.25358 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.283+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 --> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645620b2270 con 0x56456240ea10 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.284+0000 7fde11c00640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456240ea10 0x564562413a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.284+0000 7fde11c00640 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456240ea10 0x564562413a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=459e55042547f3d server_cookie=8d8a9345d36dc0f7 in_seq=0 out_seq=0 2024-03-11T19:53:07.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.284+0000 7fddf3400640 10 client.25358 ms_handle_connect on v2:172.21.15.145:6838/322644420 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fddf3400640 1 -- 172.21.15.107:0/3575435318 <== mds.2 v2:172.21.15.145:6838/322644420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645620b2270 con 0x56456240ea10 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fddf3400640 10 client.25358 handle_command_reply: tid=0 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 client.25358 shutdown 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 msgr2=0x5645624036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624036a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009620 tx=0x7fddf800f790 comp rx=0 tx=0).stop 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 shutdown_connections 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456240ea10 0x564562413a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624036a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x56456240a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 --2- 172.21.15.107:0/3575435318 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x564562403160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fddf3400640 0 client.25358 ms_handle_reset on v2:172.21.15.145:6838/322644420 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 >> 172.21.15.107:0/3575435318 conn(0x5645623f4af0 msgr2=0x5645623f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 shutdown_connections 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 1 -- 172.21.15.107:0/3575435318 wait complete. 2024-03-11T19:53:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.285+0000 7fde173fa740 20 client.25358 trim_cache size 0 max 16384 2024-03-11T19:53:07.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.302+0000 7fde173fa740 1 Processor -- start 2024-03-11T19:53:07.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.302+0000 7fde173fa740 1 -- start start 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.302+0000 7fde173fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.302+0000 7fde173fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x56456240a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde173fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x5645623f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde173fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562361840 con 0x5645623f9580 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde173fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645623f5540 con 0x5645623ff0e0 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde173fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562401700 con 0x5645623f9f70 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36550/0 (socket says 172.21.15.107:36550) 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 -- 172.21.15.107:0/2481747703 learned_addr learned my addr 172.21.15.107:0/2481747703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11200640 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x56456240a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde12600640 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x5645623f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 -- 172.21.15.107:0/2481747703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 msgr2=0x56456240a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x56456240a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 -- 172.21.15.107:0/2481747703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 msgr2=0x5645623f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x5645623f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 -- 172.21.15.107:0/2481747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645622fc8b0 con 0x5645623f9580 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.303+0000 7fde11c00640 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde0002bc70 tx=0x7fde00005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225899b34443ad45 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fddfe000640 1 -- 172.21.15.107:0/2481747703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000b8660 con 0x5645623f9580 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fddfe000640 1 -- 172.21.15.107:0/2481747703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde000f56d0 con 0x5645623f9580 2024-03-11T19:53:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fddfe000640 1 -- 172.21.15.107:0/2481747703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000bc190 con 0x5645623f9580 2024-03-11T19:53:07.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 -- 172.21.15.107:0/2481747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 msgr2=0x5645623f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde0002bc70 tx=0x7fde00005c90 comp rx=0 tx=0).stop 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 -- 172.21.15.107:0/2481747703 shutdown_connections 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623ff0e0 0x5645623f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9f70 0x56456240a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 --2- 172.21.15.107:0/2481747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623f9580 0x5645623f3580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.304+0000 7fde173fa740 1 -- 172.21.15.107:0/2481747703 >> 172.21.15.107:0/2481747703 conn(0x5645623f4af0 msgr2=0x564562418340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 -- 172.21.15.107:0/2481747703 shutdown_connections 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 -- 172.21.15.107:0/2481747703 wait complete. 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 -- start start 2024-03-11T19:53:07.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 Processor -- start 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x56456240a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.305+0000 7fde173fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x564562407820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde173fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564562410500 con 0x5645623ff0e0 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde173fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456240afa0 con 0x5645623f9f70 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde173fa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456240fea0 con 0x5645623f9580 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde173fa740 10 client.0 _ll_register_callbacks cb 0x7fddf8009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x56456240a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde12600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x564562407820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44754/0 (socket says 172.21.15.107:44754) 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 -- 172.21.15.107:0/999186924 learned_addr learned my addr 172.21.15.107:0/999186924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x56456240a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55626/0 (socket says 172.21.15.107:55626) 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 -- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 msgr2=0x56456240a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x56456240a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 -- 172.21.15.107:0/999186924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 msgr2=0x564562407820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x564562407820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 -- 172.21.15.107:0/999186924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645623294c0 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fde11200640 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddf801c460 tx=0x7fddf8005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c165e10b09a52ca server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.306+0000 7fddf3400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8017040 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fddf801c6d0 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56456240ffe0 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf80122e0 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fde173fa740 10 client.25364 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564562322510 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56456240ffe0 con 0x5645623f9f70 2024-03-11T19:53:07.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.307+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fddf80035f0 con 0x5645623f9f70 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 10 client.25364 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 10 client.25364 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 20 client.25364 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456246a9a0 0x564562466e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 4 client.25364 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 --> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564562385c40 con 0x56456246a9a0 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde11c00640 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456246a9a0 0x564562466e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fde11c00640 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456246a9a0 0x564562466e00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59d895184ce80557 server_cookie=d9e9c4ff1d65205f in_seq=0 out_seq=0 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.308+0000 7fddf3400640 10 client.25364 ms_handle_connect on v2:172.21.15.145:6838/322644420 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fddf3400640 1 -- 172.21.15.107:0/999186924 <== mds.2 v2:172.21.15.145:6838/322644420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564562385c40 con 0x56456246a9a0 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fddf3400640 10 client.25364 handle_command_reply: tid=0 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 client.25364 shutdown 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 msgr2=0x5645624071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddf801c460 tx=0x7fddf8005ce0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 shutdown_connections 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:6838/322644420,v1:172.21.15.145:6839/322644420] conn(0x56456246a9a0 0x564562466e00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645623f9f70 0x5645624071f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5645623ff0e0 0x564562407820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 --2- 172.21.15.107:0/999186924 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5645623f9580 0x56456240a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fddf3400640 0 client.25364 ms_handle_reset on v2:172.21.15.145:6838/322644420 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 >> 172.21.15.107:0/999186924 conn(0x5645623f4af0 msgr2=0x5645623f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 shutdown_connections 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 1 -- 172.21.15.107:0/999186924 wait complete. 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.309+0000 7fde173fa740 20 client.25364 trim_cache size 0 max 16384 2024-03-11T19:53:07.319 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24547, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 158.883010359, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:53:07.320 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 209.11611622199999 2024-03-11T19:53:07.321 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:53:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 msgr2=0x7fddf40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 0x7fddf40a3f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5645623fa7d0 tx=0x7fde04009500 comp rx=0 tx=0).stop 2024-03-11T19:53:07.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 msgr2=0x7fde0c19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 0x7fde0c19d820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde00036040 tx=0x7fde00005e80 comp rx=0 tx=0).stop 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 shutdown_connections 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fddf40a1ab0 0x7fddf40a3f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde0c109350 0x7fde0c1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde0c108990 0x7fde0c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 --2- 172.21.15.107:0/678334844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fde0c1077a0 0x7fde0c19d820 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 >> 172.21.15.107:0/678334844 conn(0x7fde0c076740 msgr2=0x7fde0c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 shutdown_connections 2024-03-11T19:53:07.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.310+0000 7fde14600640 1 -- 172.21.15.107:0/678334844 wait complete. 2024-03-11T19:53:07.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 -- 172.21.15.107:0/443409832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc101080 msgr2=0x7f05fc1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 --2- 172.21.15.107:0/443409832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc101080 0x7f05fc1014a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009900 tx=0x7f05ec02bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 -- 172.21.15.107:0/443409832 shutdown_connections 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 --2- 172.21.15.107:0/443409832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102c30 0x7f05fc1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 --2- 172.21.15.107:0/443409832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc102270 0x7f05fc1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 --2- 172.21.15.107:0/443409832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc101080 0x7f05fc1014a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 -- 172.21.15.107:0/443409832 >> 172.21.15.107:0/443409832 conn(0x7f05fc0fc5f0 msgr2=0x7f05fc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 -- 172.21.15.107:0/443409832 shutdown_connections 2024-03-11T19:53:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.530+0000 7f0600a00640 1 -- 172.21.15.107:0/443409832 wait complete. 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 Processor -- start 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 -- start start 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102270 0x7f05fc1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc102c30 0x7f05fc1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0e70 con 0x7f05fc101080 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0fe0 con 0x7f05fc102270 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.531+0000 7f0600a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a11a0 con 0x7f05fc102c30 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36590/0 (socket says 172.21.15.107:36590) 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 -- 172.21.15.107:0/846819469 learned_addr learned my addr 172.21.15.107:0/846819469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fb200640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc102c30 0x7f05fc1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05f9e00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102270 0x7f05fc1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 -- 172.21.15.107:0/846819469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc102c30 msgr2=0x7f05fc1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc102c30 0x7f05fc1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 -- 172.21.15.107:0/846819469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102270 msgr2=0x7f05fc1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102270 0x7f05fc1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.532+0000 7f05fa800640 1 -- 172.21.15.107:0/846819469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009560 con 0x7f05fc101080 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f05fa800640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f05ec000c00 tx=0x7f05ec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f05f3400640 1 -- 172.21.15.107:0/846819469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0044c0 con 0x7f05fc101080 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f05f3400640 1 -- 172.21.15.107:0/846819469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05ec0052d0 con 0x7f05fc101080 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f05f3400640 1 -- 172.21.15.107:0/846819469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec003e50 con 0x7f05fc101080 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1a1420 con 0x7f05fc101080 2024-03-11T19:53:07.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.533+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1a1930 con 0x7f05fc101080 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.536+0000 7f0604182740 1 Processor -- start 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.536+0000 7f05f3400640 1 -- 172.21.15.107:0/846819469 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f05ec005440 con 0x7f05fc101080 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.536+0000 7f0604182740 1 -- start start 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x55775533dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755340750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577552a7b30 con 0x557755338760 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577552a5060 con 0x55775533e2c0 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f0604182740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577552a0840 con 0x557755339150 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fa800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x55775533dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36594/0 (socket says 172.21.15.107:36594) 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 -- 172.21.15.107:0/3343490780 learned_addr learned my addr 172.21.15.107:0/3343490780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05f9e00640 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755340750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 -- 172.21.15.107:0/3343490780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 msgr2=0x55775533dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x55775533dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 -- 172.21.15.107:0/3343490780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 msgr2=0x557755340750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755340750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fb200640 1 -- 172.21.15.107:0/3343490780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577552684c0 con 0x557755338760 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.537+0000 7f05fa800640 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x55775533dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05fb200640 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c716155d59542095 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05f3400640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 0x7f05e00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05e7400640 1 -- 172.21.15.107:0/3343490780 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004d20 con 0x557755338760 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05f3400640 1 -- 172.21.15.107:0/846819469 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f05ec0f0110 con 0x7f05fc101080 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05e7400640 1 -- 172.21.15.107:0/3343490780 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05f4004e80 con 0x557755338760 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05e7400640 1 -- 172.21.15.107:0/3343490780 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4005240 con 0x557755338760 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.538+0000 7f05fa800640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 0x7f05e00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- 172.21.15.107:0/3343490780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 msgr2=0x557755338b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f05fa800640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 0x7f05e00a3ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0bca90 tx=0x7f05ec0f3b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f40118d0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- 172.21.15.107:0/3343490780 shutdown_connections 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755340750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x55775533dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 --2- 172.21.15.107:0/3343490780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x557755338b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- 172.21.15.107:0/3343490780 >> 172.21.15.107:0/3343490780 conn(0x557755333cd0 msgr2=0x5577553340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- 172.21.15.107:0/3343490780 shutdown_connections 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- 172.21.15.107:0/3343490780 wait complete. 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.539+0000 7f0604182740 1 -- start start 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 Processor -- start 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 0x557755342b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755349b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755245ea0 con 0x557755339150 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755245b40 con 0x55775533e2c0 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557754fe8f20 con 0x557755338760 2024-03-11T19:53:07.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f0604182740 10 client.0 _ll_register_callbacks cb 0x557755259950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f05fb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f05fb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55670/0 (socket says 172.21.15.107:55670) 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f05fb200640 1 -- 172.21.15.107:0/1939496074 learned_addr learned my addr 172.21.15.107:0/1939496074 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.540+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 0x557755342b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05fb200640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755349b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 -- 172.21.15.107:0/1939496074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 msgr2=0x557755342610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 -- 172.21.15.107:0/1939496074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 msgr2=0x557755349b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755349b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 -- 172.21.15.107:0/1939496074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f4009c80 con 0x557755339150 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05fb200640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 0x557755342b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05fc19a890 tx=0x7f05e800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11920cdad0cc1bfb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:53:07.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8016070 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.541+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f05e8003dc0 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557754fa06c0 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05e8005020 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 10 client.15261 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55775529eec0 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557754fa06c0 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05e801b470 con 0x557755339150 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 10 client.15261 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 10 client.15261 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 20 client.15261 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.542+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x55775534df10 0x557755352f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.543+0000 7f0604182740 4 client.15261 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.543+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 --> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557754ff1270 con 0x55775534df10 2024-03-11T19:53:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.543+0000 7f05fb200640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x55775534df10 0x557755352f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.543+0000 7f05dbe00640 10 client.15261 ms_handle_connect on v2:172.21.15.107:6840/981878935 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.543+0000 7f05fb200640 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x55775534df10 0x557755352f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7f7aa6ec9d088ec server_cookie=99ff9cd15124f291 in_seq=0 out_seq=0 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f05dbe00640 1 -- 172.21.15.107:0/1939496074 <== mds.1 v2:172.21.15.107:6840/981878935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557754ff1270 con 0x55775534df10 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f05dbe00640 10 client.15261 handle_command_reply: tid=0 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 client.15261 shutdown 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 msgr2=0x557755342b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 0x557755342b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05fc19a890 tx=0x7f05e800bf30 comp rx=0 tx=0).stop 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 shutdown_connections 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x55775534df10 0x557755352f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755349b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755339150 0x557755342b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 --2- 172.21.15.107:0/1939496074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755338760 0x557755342610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 >> 172.21.15.107:0/1939496074 conn(0x557755333cd0 msgr2=0x557755332e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.544+0000 7f05dbe00640 0 client.15261 ms_handle_reset on v2:172.21.15.107:6840/981878935 2024-03-11T19:53:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.545+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 shutdown_connections 2024-03-11T19:53:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.545+0000 7f0604182740 1 -- 172.21.15.107:0/1939496074 wait complete. 2024-03-11T19:53:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.545+0000 7f0604182740 20 client.15261 trim_cache size 0 max 16384 2024-03-11T19:53:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.562+0000 7f0604182740 1 Processor -- start 2024-03-11T19:53:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.562+0000 7f0604182740 1 -- start start 2024-03-11T19:53:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.562+0000 7f0604182740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f0604182740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x557755349ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f0604182740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755356f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f0604182740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755340e10 con 0x557755338760 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f0604182740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557754ff1270 con 0x55775533e2c0 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f0604182740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557754fa06c0 con 0x557755339150 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36608/0 (socket says 172.21.15.107:36608) 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 -- 172.21.15.107:0/3487419054 learned_addr learned my addr 172.21.15.107:0/3487419054 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05f9e00640 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x557755349ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755356f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 -- 172.21.15.107:0/3487419054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 msgr2=0x557755349ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x557755349ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 -- 172.21.15.107:0/3487419054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 msgr2=0x557755356f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755356f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 -- 172.21.15.107:0/3487419054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775523b8b0 con 0x557755338760 2024-03-11T19:53:07.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.563+0000 7f05fb200640 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05f40049d0 tx=0x7f05f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2c98cecb18e919 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f05e6a00640 1 -- 172.21.15.107:0/3487419054 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004350 con 0x557755338760 2024-03-11T19:53:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f05e6a00640 1 -- 172.21.15.107:0/3487419054 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05f4011e00 con 0x557755338760 2024-03-11T19:53:07.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f05e6a00640 1 -- 172.21.15.107:0/3487419054 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4028450 con 0x557755338760 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 -- 172.21.15.107:0/3487419054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 msgr2=0x55775533d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05f40049d0 tx=0x7f05f40118d0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 -- 172.21.15.107:0/3487419054 shutdown_connections 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55775533e2c0 0x557755356f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557755339150 0x557755349ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 --2- 172.21.15.107:0/3487419054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x55775533d900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.564+0000 7f0604182740 1 -- 172.21.15.107:0/3487419054 >> 172.21.15.107:0/3487419054 conn(0x557755333cd0 msgr2=0x557755332e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.565+0000 7f0604182740 1 -- 172.21.15.107:0/3487419054 shutdown_connections 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.565+0000 7f0604182740 1 -- 172.21.15.107:0/3487419054 wait complete. 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.565+0000 7f0604182740 1 -- start start 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.565+0000 7f0604182740 1 Processor -- start 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 0x557755345f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775533e2c0 0x557755348f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755349490 con 0x557755338760 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557755349600 con 0x557755339150 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577553497c0 con 0x55775533e2c0 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f0604182740 10 client.0 _ll_register_callbacks cb 0x7f05e8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 0x557755345f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05fb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36614/0 (socket says 172.21.15.107:36614) 2024-03-11T19:53:07.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05fb200640 1 -- 172.21.15.107:0/1272788118 learned_addr learned my addr 172.21.15.107:0/1272788118 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05fb200640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775533e2c0 0x557755348f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 -- 172.21.15.107:0/1272788118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775533e2c0 msgr2=0x557755348f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775533e2c0 0x557755348f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 -- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 msgr2=0x5577553459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05f9e00640 1 -- 172.21.15.107:0/1272788118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577552684c0 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.566+0000 7f05fb200640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05f9e00640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 0x557755345f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05e8001eb0 tx=0x7f05e8005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6dfac5231d87e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8016040 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55775534f4a0 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f05e8000e40 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05e801b470 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f0604182740 10 client.25379 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557755261510 con 0x557755339150 2024-03-11T19:53:07.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.567+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55775534f4a0 con 0x557755339150 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05e8004600 con 0x557755339150 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 10 client.25379 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 10 client.25379 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 20 client.25379 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5577553a95d0 0x5577553a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 4 client.25379 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 --> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5577552c4c40 con 0x5577553a95d0 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f05fb200640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5577553a95d0 0x5577553a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f05fb200640 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5577553a95d0 0x5577553a5a50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f910365d4aa9cb4 server_cookie=b7aabd8ef65dc1ec in_seq=0 out_seq=0 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.568+0000 7f05dbe00640 10 client.25379 ms_handle_connect on v2:172.21.15.107:6840/981878935 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f05dbe00640 1 -- 172.21.15.107:0/1272788118 <== mds.1 v2:172.21.15.107:6840/981878935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5577552c4c40 con 0x5577553a95d0 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f05dbe00640 10 client.25379 handle_command_reply: tid=0 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 client.25379 shutdown 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 msgr2=0x557755345f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 0x557755345f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05e8001eb0 tx=0x7f05e8005cc0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 shutdown_connections 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935] conn(0x5577553a95d0 0x5577553a5a50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557755339150 0x557755345f30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55775533e2c0 0x557755348f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f05dbe00640 0 client.25379 ms_handle_reset on v2:172.21.15.107:6840/981878935 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 --2- 172.21.15.107:0/1272788118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557755338760 0x5577553459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 >> 172.21.15.107:0/1272788118 conn(0x557755333cd0 msgr2=0x557755332e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 shutdown_connections 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 1 -- 172.21.15.107:0/1272788118 wait complete. 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.569+0000 7f0604182740 20 client.25379 trim_cache size 0 max 16384 2024-03-11T19:53:07.578 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24583, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 160.15578609400001, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 201.11069186 2024-03-11T19:53:07.579 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 msgr2=0x7f05e00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 0x7f05e00a3ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0bca90 tx=0x7f05ec0f3b10 comp rx=0 tx=0).stop 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 msgr2=0x7f05fc199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f05ec000c00 tx=0x7f05ec0040c0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 shutdown_connections 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f05e00a1a30 0x7f05e00a3ef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05fc102270 0x7f05fc1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05fc102c30 0x7f05fc1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 --2- 172.21.15.107:0/846819469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc101080 0x7f05fc199380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 >> 172.21.15.107:0/846819469 conn(0x7f05fc0fc5f0 msgr2=0x7f05fc0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 shutdown_connections 2024-03-11T19:53:07.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.570+0000 7f0600a00640 1 -- 172.21.15.107:0/846819469 wait complete. 2024-03-11T19:53:07.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:07 smithi107 ceph-mon[31712]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:53:07.591 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:07 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1623523263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:07.591 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:07 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/162194451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:07.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:53:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:07 smithi145 ceph-mon[36730]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:53:07.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:07 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1623523263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:07.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:07 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/162194451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 -- 172.21.15.107:0/2871569622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd9010bda0 msgr2=0x7efd9010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 --2- 172.21.15.107:0/2871569622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd9010bda0 0x7efd9010e190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009900 tx=0x7efd7c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 -- 172.21.15.107:0/2871569622 shutdown_connections 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 --2- 172.21.15.107:0/2871569622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd9010bda0 0x7efd9010e190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 --2- 172.21.15.107:0/2871569622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010e6d0 0x7efd90110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 --2- 172.21.15.107:0/2871569622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd90069f90 0x7efd901083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 -- 172.21.15.107:0/2871569622 >> 172.21.15.107:0/2871569622 conn(0x7efd9006e380 msgr2=0x7efd900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 -- 172.21.15.107:0/2871569622 shutdown_connections 2024-03-11T19:53:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.791+0000 7efd95200640 1 -- 172.21.15.107:0/2871569622 wait complete. 2024-03-11T19:53:07.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.792+0000 7efd95200640 1 Processor -- start 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.792+0000 7efd95200640 1 -- start start 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 0x7efd9019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9010e6d0 0x7efd901a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901a75a0 con 0x7efd9010bda0 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901a7710 con 0x7efd90069f90 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd95200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901a78d0 con 0x7efd9010e6d0 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd8d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd8d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36636/0 (socket says 172.21.15.107:36636) 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd8d400640 1 -- 172.21.15.107:0/2368030095 learned_addr learned my addr 172.21.15.107:0/2368030095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd8de00640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 0x7efd9019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.793+0000 7efd8e800640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9010e6d0 0x7efd901a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 -- 172.21.15.107:0/2368030095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9010e6d0 msgr2=0x7efd901a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9010e6d0 0x7efd901a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 -- 172.21.15.107:0/2368030095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 msgr2=0x7efd9019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 0x7efd9019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 -- 172.21.15.107:0/2368030095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80009490 con 0x7efd9010bda0 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8de00640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 0x7efd9019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd8d400640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000c00 tx=0x7efd7c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd86a00640 1 -- 172.21.15.107:0/2368030095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0043d0 con 0x7efd9010bda0 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd86a00640 1 -- 172.21.15.107:0/2368030095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd7c0052d0 con 0x7efd9010bda0 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c009560 con 0x7efd9010bda0 2024-03-11T19:53:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd86a00640 1 -- 172.21.15.107:0/2368030095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003da0 con 0x7efd9010bda0 2024-03-11T19:53:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.794+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd901a7f20 con 0x7efd9010bda0 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd86a00640 1 -- 172.21.15.107:0/2368030095 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7efd7c005440 con 0x7efd9010bda0 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 Processor -- start 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 -- start start 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e64f0 0x55c5404e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54044bb30 con 0x55c5404e1380 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c540449060 con 0x55c5404e0990 2024-03-11T19:53:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd97e83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c540444840 con 0x55c5404e64f0 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.798+0000 7efd8de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36642/0 (socket says 172.21.15.107:36642) 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e64f0 0x55c5404e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 -- 172.21.15.107:0/1995268831 learned_addr learned my addr 172.21.15.107:0/1995268831 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8e800640 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 -- 172.21.15.107:0/1995268831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e64f0 msgr2=0x55c5404e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e64f0 0x55c5404e8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 -- 172.21.15.107:0/1995268831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 msgr2=0x55c5404e0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 -- 172.21.15.107:0/1995268831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c54040c4c0 con 0x55c5404e1380 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd86a00640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 0x7efd740a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.799+0000 7efd8de00640 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efd80011680 tx=0x7efd80011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd4d6bc0968e83e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.800+0000 7efd86a00640 1 -- 172.21.15.107:0/2368030095 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd7c0efff0 con 0x7efd9010bda0 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.800+0000 7efd8e800640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 0x7efd740a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.800+0000 7efd7b400640 1 -- 172.21.15.107:0/1995268831 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000fba0 con 0x55c5404e1380 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.800+0000 7efd7b400640 1 -- 172.21.15.107:0/1995268831 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd8000fd00 con 0x55c5404e1380 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.800+0000 7efd7b400640 1 -- 172.21.15.107:0/1995268831 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000e6e0 con 0x55c5404e1380 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- 172.21.15.107:0/1995268831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 msgr2=0x55c5404e5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efd80011680 tx=0x7efd80011b40 comp rx=0 tx=0).stop 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd8e800640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 0x7efd740a3f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55c5404e1be0 tx=0x7efd88009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- 172.21.15.107:0/1995268831 shutdown_connections 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e64f0 0x55c5404e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404e5e80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 --2- 172.21.15.107:0/1995268831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- 172.21.15.107:0/1995268831 >> 172.21.15.107:0/1995268831 conn(0x55c5404d7af0 msgr2=0x55c5404d7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- 172.21.15.107:0/1995268831 shutdown_connections 2024-03-11T19:53:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- 172.21.15.107:0/1995268831 wait complete. 2024-03-11T19:53:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 -- start start 2024-03-11T19:53:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.801+0000 7efd97e83740 1 Processor -- start 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 0x55c5404ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 0x55c5404f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5403e9ea0 con 0x55c5404e1380 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5403e9b40 con 0x55c5404e64f0 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54018cf20 con 0x55c5404e0990 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd97e83740 10 client.0 _ll_register_callbacks cb 0x55c5403fd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd8d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 0x55c5404ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd8e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 0x55c5404f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 0x55c5404ea7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55714/0 (socket says 172.21.15.107:55714) 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.802+0000 7efd8de00640 1 -- 172.21.15.107:0/1244214387 learned_addr learned my addr 172.21.15.107:0/1244214387 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 -- 172.21.15.107:0/1244214387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 msgr2=0x55c5404ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 0x55c5404ea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 -- 172.21.15.107:0/1244214387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 msgr2=0x55c5404ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 -- 172.21.15.107:0/1244214387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd80009e50 con 0x55c5404e64f0 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8d400640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd8e800640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 0x55c5404f1d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd88005fd0 tx=0x7efd88004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec30f4bbd366b6c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88016070 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd8800f7b0 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.803+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c540468c40 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8800f970 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 10 client.25391 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c540442ec0 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c540468c40 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd88028d00 con 0x55c5404e64f0 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 10 client.25391 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 10 client.25391 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 20 client.25391 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404f6e80 0x55c5404f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 4 client.25391 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.804+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5401446c0 con 0x55c5404f6e80 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.805+0000 7efd8de00640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404f6e80 0x55c5404f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.805+0000 7efd8de00640 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404f6e80 0x55c5404f60f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81add9217291ff3 server_cookie=1db510d73df584eb in_seq=0 out_seq=0 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.805+0000 7efd6fe00640 10 client.25391 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.806+0000 7efd6fe00640 1 -- 172.21.15.107:0/1244214387 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5401446c0 con 0x55c5404f6e80 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.806+0000 7efd6fe00640 10 client.25391 handle_command_reply: tid=0 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.806+0000 7efd97e83740 1 client.25391 shutdown 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.806+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 msgr2=0x55c5404f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.806+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 0x55c5404f1d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd88005fd0 tx=0x7efd88004ab0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 shutdown_connections 2024-03-11T19:53:07.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404f6e80 0x55c5404f60f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e64f0 0x55c5404f1d60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 --2- 172.21.15.107:0/1244214387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e0990 0x55c5404ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd6fe00640 0 client.25391 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 >> 172.21.15.107:0/1244214387 conn(0x55c5404d7af0 msgr2=0x55c5404d6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 shutdown_connections 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 1 -- 172.21.15.107:0/1244214387 wait complete. 2024-03-11T19:53:07.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.807+0000 7efd97e83740 20 client.25391 trim_cache size 0 max 16384 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 Processor -- start 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 -- start start 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e1380 0x55c5404e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5404d8540 con 0x55c5404d7af0 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5401446c0 con 0x55c5404e0990 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.824+0000 7efd97e83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c540468c40 con 0x55c5404e1380 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36660/0 (socket says 172.21.15.107:36660) 2024-03-11T19:53:07.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 -- 172.21.15.107:0/981769874 learned_addr learned my addr 172.21.15.107:0/981769874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8e800640 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e1380 0x55c5404e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8d400640 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 -- 172.21.15.107:0/981769874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e1380 msgr2=0x55c5404e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e1380 0x55c5404e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 -- 172.21.15.107:0/981769874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 msgr2=0x55c5404e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 -- 172.21.15.107:0/981769874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5403df8b0 con 0x55c5404d7af0 2024-03-11T19:53:07.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.825+0000 7efd8de00640 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efd80011680 tx=0x7efd80005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65d475e727ec511 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd7aa00640 1 -- 172.21.15.107:0/981769874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800052c0 con 0x55c5404d7af0 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd7aa00640 1 -- 172.21.15.107:0/981769874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd80019910 con 0x55c5404d7af0 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd7aa00640 1 -- 172.21.15.107:0/981769874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80003850 con 0x55c5404d7af0 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 -- 172.21.15.107:0/981769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 msgr2=0x55c5404e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efd80011680 tx=0x7efd80005850 comp rx=0 tx=0).stop 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 -- 172.21.15.107:0/981769874 shutdown_connections 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404e1380 0x55c5404e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 --2- 172.21.15.107:0/981769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404d7af0 0x55c5404e0160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 -- 172.21.15.107:0/981769874 >> 172.21.15.107:0/981769874 conn(0x55c5404d3010 msgr2=0x55c5404ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 -- 172.21.15.107:0/981769874 shutdown_connections 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.826+0000 7efd97e83740 1 -- 172.21.15.107:0/981769874 wait complete. 2024-03-11T19:53:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 -- start start 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 Processor -- start 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 0x55c5404dece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404df220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5404df900 con 0x55c5404e1380 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5404dfa70 con 0x55c5404e0990 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c540551270 con 0x55c5404d7af0 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.827+0000 7efd97e83740 10 client.0 _ll_register_callbacks cb 0x7efd88009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 0x55c5404dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36662/0 (socket says 172.21.15.107:36662) 2024-03-11T19:53:07.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8e800640 1 -- 172.21.15.107:0/186255861 learned_addr learned my addr 172.21.15.107:0/186255861 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 -- 172.21.15.107:0/186255861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 msgr2=0x55c5404dece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 0x55c5404dece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 -- 172.21.15.107:0/186255861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 msgr2=0x55c5404f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 -- 172.21.15.107:0/186255861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c54040c4c0 con 0x55c5404e0990 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8e800640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8de00640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 0x55c5404dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd8d400640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404df220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0bcc40 tx=0x7efd7c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd76afd92064d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.828+0000 7efd6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0bca80 con 0x55c5404e0990 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd7c0bd3b0 con 0x55c5404e0990 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5404f7eb0 con 0x55c5404e0990 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd7c0bc7f0 con 0x55c5404e0990 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 10 client.25397 fetch_fsmap learned FSMap version 23 2024-03-11T19:53:07.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c540405510 con 0x55c5404e0990 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5404f7eb0 con 0x55c5404e0990 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 23) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd7c03c070 con 0x55c5404e0990 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 10 client.25397 fetch_fsmap finished waiting for FSMap version 23 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 10 client.25397 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 20 client.25397 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404e64f0 0x55c540552760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 4 client.25397 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.829+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5404f6b90 con 0x55c5404e64f0 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd8de00640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404e64f0 0x55c540552760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd8de00640 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404e64f0 0x55c540552760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f8c74d47e7b419 server_cookie=76a6d4b22b12950d in_seq=0 out_seq=0 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd6fe00640 10 client.25397 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd6fe00640 1 -- 172.21.15.107:0/186255861 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c5404f6b90 con 0x55c5404e64f0 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd6fe00640 10 client.25397 handle_command_reply: tid=0 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.830+0000 7efd97e83740 1 client.25397 shutdown 2024-03-11T19:53:07.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 msgr2=0x55c5404df220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404df220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0bcc40 tx=0x7efd7c0045f0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 shutdown_connections 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c5404e64f0 0x55c540552760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5404e0990 0x55c5404df220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5404e1380 0x55c5404f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd6fe00640 0 client.25397 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 --2- 172.21.15.107:0/186255861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c5404d7af0 0x55c5404dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 >> 172.21.15.107:0/186255861 conn(0x55c5404d3010 msgr2=0x55c5404e9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 shutdown_connections 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 1 -- 172.21.15.107:0/186255861 wait complete. 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd97e83740 20 client.25397 trim_cache size 0 max 16384 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1ccf666c-dfdf-11ee-95c8-87774f69a715", 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-03-11T19:53:07.842 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24590, 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 191.65793689099999, 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 95, 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 90, 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 198.34521362300001 2024-03-11T19:53:07.843 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T19:53:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 msgr2=0x7efd740a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 0x7efd740a3f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55c5404e1be0 tx=0x7efd88009500 comp rx=0 tx=0).stop 2024-03-11T19:53:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 msgr2=0x7efd9019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.831+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000c00 tx=0x7efd7c005e80 comp rx=0 tx=0).stop 2024-03-11T19:53:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 shutdown_connections 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7efd740a1ab0 0x7efd740a3f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd9010e6d0 0x7efd901a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd9010bda0 0x7efd9019fff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 --2- 172.21.15.107:0/2368030095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd90069f90 0x7efd9019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 >> 172.21.15.107:0/2368030095 conn(0x7efd9006e380 msgr2=0x7efd90109180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 shutdown_connections 2024-03-11T19:53:07.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:07.832+0000 7efd95200640 1 -- 172.21.15.107:0/2368030095 wait complete. 2024-03-11T19:53:07.853 INFO:teuthology.run_tasks:Running task kclient... 2024-03-11T19:53:07.866 INFO:tasks.kclient:Mounting kernel clients... 2024-03-11T19:53:07.867 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-11T19:53:07.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:07.867 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-11T19:53:07.916 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-11T19:53:07.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:07.916 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T19:53:08.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:08.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-03-11T19:53:08.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:07 smithi122 ceph-mon[36397]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:53:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:07 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1623523263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:08.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:07 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/162194451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:53:08.140 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-11T19:53:08.141 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T19:53:08.141 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:08.141 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-11T19:53:08.571 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:53:09.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:09 smithi145 ceph-mon[36730]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:09.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:09 smithi107 ceph-mon[31712]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:10.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:09 smithi122 ceph-mon[36397]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:10.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.835+0000 7f5bb3000700 1 -- 172.21.15.107:0/3620637135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1040c0 msgr2=0x7f5bac1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.835+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3620637135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1040c0 0x7f5bac1044a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009990 tx=0x7f5b9c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 -- 172.21.15.107:0/3620637135 shutdown_connections 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3620637135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1091a0 0x7f5bac110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3620637135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1040c0 0x7f5bac1044a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3620637135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 -- 172.21.15.107:0/3620637135 >> 172.21.15.107:0/3620637135 conn(0x7f5bac0ff950 msgr2=0x7f5bac101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 -- 172.21.15.107:0/3620637135 shutdown_connections 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.836+0000 7f5bb3000700 1 -- 172.21.15.107:0/3620637135 wait complete. 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.837+0000 7f5bb3000700 1 Processor -- start 2024-03-11T19:53:10.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.837+0000 7f5bb3000700 1 -- start start 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.837+0000 7f5bb3000700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.837+0000 7f5bb3000700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.837+0000 7f5bb3000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1091a0 0x7f5bac19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bb3000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19f2a0 con 0x7f5bac1091a0 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bb3000700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19f410 con 0x7f5bac104a70 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bb3000700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19f5d0 con 0x7f5bac1040c0 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bab400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bab400700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44856/0 (socket says 172.21.15.107:44856) 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:55768/0 (socket says 172.21.15.107:55768) 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bab400700 1 -- 172.21.15.107:0/3551379093 learned_addr learned my addr 172.21.15.107:0/3551379093 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bb1000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1091a0 0x7f5bac19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:10.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 -- 172.21.15.107:0/3551379093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 msgr2=0x7f5bac19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 -- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1091a0 msgr2=0x7f5bac19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1091a0 0x7f5bac19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5babe00700 1 -- 172.21.15.107:0/3551379093 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0095f0 con 0x7f5bac1040c0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.838+0000 7f5bab400700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5babe00700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000c00 tx=0x7f5b9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c01d070 con 0x7f5bac1040c0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b9c00b610 con 0x7f5bac1040c0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c021e80 con 0x7f5bac1040c0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac19f850 con 0x7f5bac1040c0 2024-03-11T19:53:10.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.839+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac1a7110 con 0x7f5bac1040c0 2024-03-11T19:53:10.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.840+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac10c460 con 0x7f5bac1040c0 2024-03-11T19:53:10.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.842+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f5b9c021580 con 0x7f5bac1040c0 2024-03-11T19:53:10.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.843+0000 7f5ba8c00700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 0x7f5b900a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:10.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.844+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5b9c0ce8d0 con 0x7f5bac1040c0 2024-03-11T19:53:10.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.844+0000 7f5bab400700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 0x7f5b900a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:10.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.844+0000 7f5bab400700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 0x7f5b900a3f80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5bac10a770 tx=0x7f5b98007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:10.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:10.846+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b9c097760 con 0x7f5bac1040c0 2024-03-11T19:53:11.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.110+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5bac1055a0 con 0x7f5bac1040c0 2024-03-11T19:53:11.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.111+0000 7f5ba8c00700 1 -- 172.21.15.107:0/3551379093 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5b9c09ae20 con 0x7f5bac1040c0 2024-03-11T19:53:11.116 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 msgr2=0x7f5b900a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 0x7f5b900a3f80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5bac10a770 tx=0x7f5b98007480 comp rx=0 tx=0).stop 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 msgr2=0x7f5bac19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000c00 tx=0x7f5b9c004730 comp rx=0 tx=0).stop 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 shutdown_connections 2024-03-11T19:53:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f5b900a1ac0 0x7f5b900a3f80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bac1091a0 0x7f5bac19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bac1040c0 0x7f5bac19a410 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 --2- 172.21.15.107:0/3551379093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bac104a70 0x7f5bac19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 >> 172.21.15.107:0/3551379093 conn(0x7f5bac0ff950 msgr2=0x7f5bac101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 shutdown_connections 2024-03-11T19:53:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:11.114+0000 7f5bb3000700 1 -- 172.21.15.107:0/3551379093 wait complete. 2024-03-11T19:53:11.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:11 smithi145 ceph-mon[36730]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:11.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:11 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3551379093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:53:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:11 smithi107 ceph-mon[31712]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:11.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:11 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3551379093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:11.910 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-11T19:53:11.911 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T19:53:11.911 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T19:53:11.911 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-11T19:53:11.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:11.911 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-03-11T19:53:11.966 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41875sec preferred_lft 41875sec 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-11T19:53:11.967 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:53:11.968 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T19:53:11.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:11.968 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:53:11.968 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:53:11.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-03-11T19:53:11.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-03-11T19:53:11.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-03-11T19:53:11.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T19:53:11.969 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:53:12.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:11 smithi122 ceph-mon[36397]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:53:12.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:11 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3551379093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:53:12.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.184 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T19:53:12.258 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-03-11T19:53:12.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.262 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-03-11T19:53:12.314 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-03-11T19:53:12.315 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T19:53:12.315 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T19:53:12.315 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T19:53:12.315 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T19:53:12.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T19:53:12.316 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:53:12.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.504 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-11T19:53:12.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.554 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-03-11T19:53:12.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.604 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:53:12.604 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:53:12.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T19:53:12.605 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-11T19:53:12.605 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:53:12.740 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-11T19:53:12.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.740 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:53:12.740 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:53:12.740 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-11T19:53:12.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T19:53:12.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-11T19:53:12.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-11T19:53:12.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-11T19:53:12.745 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:53:12.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:12.988 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-11T19:53:12.988 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-11T19:53:12.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-03-11T19:53:12.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T19:53:12.988 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-11T19:53:13.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:53:13.154 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:13.200 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-11T19:53:13.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:53:13.200 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:13.295 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-11T19:53:13.296 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-11T19:53:13.296 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/1159a975-ef10-422d-92cf-d6ce759e4d5b 2024-03-11T19:53:13.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:13.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/1159a975-ef10-422d-92cf-d6ce759e4d5b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:53:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:53:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-11T19:53:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-03-11T19:53:13.741 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:13 smithi107 ceph-mon[31712]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:14.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:13 smithi122 ceph-mon[36397]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:14.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:13 smithi145 ceph-mon[36730]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.122:3300,172.21.15.145:3300" 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.107:3300,172.21.15.122:3300,172.21.15.145:3300:/volumes/qa/sv_1/1159a975-ef10-422d-92cf-d6ce759e4d5b 2024-03-11T19:53:14.124 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=1ccf666c-dfdf-11ee-95c8-87774f69a715" will pass to kernel 2024-03-11T19:53:14.127 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-11T19:53:14.127 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-03-11T19:53:14.127 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-11T19:53:14.127 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-03-11T19:53:14.128 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-11T19:53:14.133 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-11T19:53:14.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:14.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:14.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:14.225 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:53:14.281 INFO:teuthology.orchestra.run.smithi107.stdout:client25411 2024-03-11T19:53:14.281 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:53:14.281 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:53:14.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:14.333 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:53:14.386 INFO:teuthology.orchestra.run.smithi107.stdout:client25411 2024-03-11T19:53:14.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:14.387 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:53:14.437 INFO:teuthology.orchestra.run.smithi107.stdout:client25411 2024-03-11T19:53:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:14.437 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715.client25411/status') 2024-03-11T19:53:14.525 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.25411 (3)172.21.15.107:0/3836107539 2024-03-11T19:53:14.525 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-03-11T19:53:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-03-11T19:53:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-03-11T19:53:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:72 bytes copied, 0.000142735 s, 504 kB/s 2024-03-11T19:53:14.530 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:53:14.541 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:53:14.541 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:53:14.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-11T19:53:14.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 -- 172.21.15.107:0/1863766128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a41033e0 msgr2=0x7f98a4103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:14.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 --2- 172.21.15.107:0/1863766128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a41033e0 0x7f98a4103800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989402bc50 comp rx=0 tx=0).stop 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 -- 172.21.15.107:0/1863766128 shutdown_connections 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 --2- 172.21.15.107:0/1863766128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41045d0 0x7f98a4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 --2- 172.21.15.107:0/1863766128 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a4104f90 0x7f98a410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 --2- 172.21.15.107:0/1863766128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a41033e0 0x7f98a4103800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.744+0000 7f98aa800640 1 -- 172.21.15.107:0/1863766128 >> 172.21.15.107:0/1863766128 conn(0x7f98a40fe970 msgr2=0x7f98a4100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.745+0000 7f98aa800640 1 -- 172.21.15.107:0/1863766128 shutdown_connections 2024-03-11T19:53:14.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.745+0000 7f98aa800640 1 -- 172.21.15.107:0/1863766128 wait complete. 2024-03-11T19:53:14.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 Processor -- start 2024-03-11T19:53:14.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 -- start start 2024-03-11T19:53:14.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a41045d0 0x7f98a4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a4104f90 0x7f98a41a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41a0ec0 con 0x7f98a4104f90 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41a1030 con 0x7f98a41033e0 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.746+0000 7f98aa800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41a11f0 con 0x7f98a41045d0 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44870/0 (socket says 172.21.15.107:44870) 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 -- 172.21.15.107:0/664752945 learned_addr learned my addr 172.21.15.107:0/664752945 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a2a00640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a41045d0 0x7f98a4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3e00640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a4104f90 0x7f98a41a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 -- 172.21.15.107:0/664752945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a41045d0 msgr2=0x7f98a4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a41045d0 0x7f98a4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 -- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a4104f90 msgr2=0x7f98a41a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:14.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a4104f90 0x7f98a41a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 -- 172.21.15.107:0/664752945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009490 con 0x7f98a41033e0 2024-03-11T19:53:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.747+0000 7f98a3400640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9894005ab0 tx=0x7f9894005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.748+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940043d0 con 0x7f98a41033e0 2024-03-11T19:53:14.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.748+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9894009560 con 0x7f98a41033e0 2024-03-11T19:53:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.748+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98940052d0 con 0x7f98a41033e0 2024-03-11T19:53:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.748+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894003d40 con 0x7f98a41033e0 2024-03-11T19:53:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.748+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a41a17f0 con 0x7f98a41033e0 2024-03-11T19:53:14.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.749+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a410a6a0 con 0x7f98a41033e0 2024-03-11T19:53:14.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.751+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7f9894005440 con 0x7f98a41033e0 2024-03-11T19:53:14.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.752+0000 7f989be00640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 0x7f98840a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.752+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f98940efbe0 con 0x7f98a41033e0 2024-03-11T19:53:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.752+0000 7f98a2a00640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 0x7f98840a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:14.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.753+0000 7f98a2a00640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 0x7f98840a3f90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98a419a8e0 tx=0x7f988c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:14.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.755+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f98940b89e0 con 0x7f98a41033e0 2024-03-11T19:53:14.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:14.990+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f98a410edf0 con 0x7f98a41033e0 2024-03-11T19:53:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.555+0000 7f989be00640 1 -- 172.21.15.107:0/664752945 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f98940bc0a0 con 0x7f98a41033e0 2024-03-11T19:53:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 msgr2=0x7f98840a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 0x7f98840a3f90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98a419a8e0 tx=0x7f988c005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 msgr2=0x7f98a41993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9894005ab0 tx=0x7f9894005e50 comp rx=0 tx=0).stop 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 shutdown_connections 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f98840a1ad0 0x7f98840a3f90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a4104f90 0x7f98a41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f98a41045d0 0x7f98a4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.558+0000 7f98aa800640 1 --2- 172.21.15.107:0/664752945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98a41033e0 0x7f98a41993d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.559+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 >> 172.21.15.107:0/664752945 conn(0x7f98a40fe970 msgr2=0x7f98a4107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.559+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 shutdown_connections 2024-03-11T19:53:15.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.559+0000 7f98aa800640 1 -- 172.21.15.107:0/664752945 wait complete. 2024-03-11T19:53:15.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-11T19:53:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 -- 172.21.15.107:0/707023305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b01033e0 msgr2=0x7fb7b0103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/707023305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b01033e0 0x7fb7b0103800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009be0 tx=0x7fb7a002bef0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 -- 172.21.15.107:0/707023305 shutdown_connections 2024-03-11T19:53:15.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/707023305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01045d0 0x7fb7b0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/707023305 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b0104f90 0x7fb7b010b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/707023305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b01033e0 0x7fb7b0103800 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 -- 172.21.15.107:0/707023305 >> 172.21.15.107:0/707023305 conn(0x7fb7b00fe970 msgr2=0x7fb7b0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 -- 172.21.15.107:0/707023305 shutdown_connections 2024-03-11T19:53:15.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.790+0000 7fb7b6e00640 1 -- 172.21.15.107:0/707023305 wait complete. 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.791+0000 7fb7b6e00640 1 Processor -- start 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.791+0000 7fb7b6e00640 1 -- start start 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 0x7fb7b0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b01045d0 0x7fb7b0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:15.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b01a0ed0 con 0x7fb7b0104f90 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b01a1040 con 0x7fb7b01033e0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b6e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b01a1200 con 0x7fb7b01045d0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34684/0 (socket says 172.21.15.107:34684) 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7b4e00640 1 -- 172.21.15.107:0/3426197930 learned_addr learned my addr 172.21.15.107:0/3426197930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7afe00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 0x7fb7b0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.792+0000 7fb7af400640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b01045d0 0x7fb7b0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 -- 172.21.15.107:0/3426197930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b01045d0 msgr2=0x7fb7b0199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b01045d0 0x7fb7b0199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 -- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 msgr2=0x7fb7b01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 -- 172.21.15.107:0/3426197930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009560 con 0x7fb7b01033e0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7b4e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7afe00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 0x7fb7b0199430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0036040 tx=0x7fb7a0033b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0003b70 con 0x7fb7b01033e0 2024-03-11T19:53:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0003cd0 con 0x7fb7b01033e0 2024-03-11T19:53:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a0009840 con 0x7fb7b01033e0 2024-03-11T19:53:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0034930 con 0x7fb7b01033e0 2024-03-11T19:53:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.793+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b01a1770 con 0x7fb7b01033e0 2024-03-11T19:53:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.794+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b010a700 con 0x7fb7b01033e0 2024-03-11T19:53:15.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.796+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fb7a00045b0 con 0x7fb7b01033e0 2024-03-11T19:53:15.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.798+0000 7fb7acc00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 0x7fb7940a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:15.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.798+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb7a00f01e0 con 0x7fb7b01033e0 2024-03-11T19:53:15.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.798+0000 7fb7af400640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 0x7fb7940a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:15.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.799+0000 7fb7af400640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 0x7fb7940a3f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7b019a8f0 tx=0x7fb798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:15.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:15.802+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7a00b9090 con 0x7fb7b01033e0 2024-03-11T19:53:16.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:15 smithi122 ceph-mon[36397]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:16.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/664752945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:15 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.029+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7b002cee0 con 0x7fb7940a1ad0 2024-03-11T19:53:16.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:15 smithi145 ceph-mon[36730]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:16.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/664752945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:15 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:15 smithi107 ceph-mon[31712]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:53:16.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/664752945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:15 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-11T19:53:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.055+0000 7fb7acc00640 1 -- 172.21.15.107:0/3426197930 <== mgr.14152 v2:172.21.15.107:6800/1546798510 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fb7b002cee0 con 0x7fb7940a1ad0 2024-03-11T19:53:16.060 INFO:teuthology.orchestra.run.smithi107.stdout:{} 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 msgr2=0x7fb7940a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 0x7fb7940a3f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7b019a8f0 tx=0x7fb798009260 comp rx=0 tx=0).stop 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 msgr2=0x7fb7b0199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 0x7fb7b0199430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0036040 tx=0x7fb7a0033b00 comp rx=0 tx=0).stop 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 shutdown_connections 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fb7940a1ad0 0x7fb7940a3f90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7b0104f90 0x7fb7b01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb7b01045d0 0x7fb7b0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 --2- 172.21.15.107:0/3426197930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b01033e0 0x7fb7b0199430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 >> 172.21.15.107:0/3426197930 conn(0x7fb7b00fe970 msgr2=0x7fb7b01078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 shutdown_connections 2024-03-11T19:53:16.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.058+0000 7fb7b6e00640 1 -- 172.21.15.107:0/3426197930 wait complete. 2024-03-11T19:53:16.081 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-11T19:53:16.092 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:53:16.100 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:53:16.100 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:53:16.100 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-11T19:53:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 -- 172.21.15.107:0/395578727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 msgr2=0x7fe87c10fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 --2- 172.21.15.107:0/395578727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c10fa40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe874009be0 tx=0x7fe87402bef0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 -- 172.21.15.107:0/395578727 shutdown_connections 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 --2- 172.21.15.107:0/395578727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c10fa40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 --2- 172.21.15.107:0/395578727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c1087f0 0x7fe87c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 --2- 172.21.15.107:0/395578727 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c107600 0x7fe87c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 -- 172.21.15.107:0/395578727 >> 172.21.15.107:0/395578727 conn(0x7fe87c0765c0 msgr2=0x7fe87c0789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 -- 172.21.15.107:0/395578727 shutdown_connections 2024-03-11T19:53:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.282+0000 7fe880a00640 1 -- 172.21.15.107:0/395578727 wait complete. 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.283+0000 7fe880a00640 1 Processor -- start 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.283+0000 7fe880a00640 1 -- start start 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1087f0 0x7fe87c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1a5120 con 0x7fe87c107600 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1a5290 con 0x7fe87c1091b0 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe880a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1a5450 con 0x7fe87c1087f0 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34722/0 (socket says 172.21.15.107:34722) 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 -- 172.21.15.107:0/1386129622 learned_addr learned my addr 172.21.15.107:0/1386129622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe871e00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1087f0 0x7fe87c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87b200640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 -- 172.21.15.107:0/1386129622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1087f0 msgr2=0x7fe87c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1087f0 0x7fe87c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 -- 172.21.15.107:0/1386129622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 msgr2=0x7fe87c1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.284+0000 7fe87a800640 1 -- 172.21.15.107:0/1386129622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874009840 con 0x7fe87c107600 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe87a800640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00e9f0 tx=0x7fe86c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00ccc0 con 0x7fe87c107600 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe86c00ce20 con 0x7fe87c107600 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c010430 con 0x7fe87c107600 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c1a5730 con 0x7fe87c107600 2024-03-11T19:53:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.285+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c1a5c40 con 0x7fe87c107600 2024-03-11T19:53:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.286+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c10e920 con 0x7fe87c107600 2024-03-11T19:53:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.288+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 136280+0+0 (secure 0 0 0) 0x7fe86c003fb0 con 0x7fe87c107600 2024-03-11T19:53:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.289+0000 7fe873e00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 0x7fe8600a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.290+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe86c014070 con 0x7fe87c107600 2024-03-11T19:53:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.290+0000 7fe871e00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 0x7fe8600a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.290+0000 7fe871e00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 0x7fe8600a3ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe87c19eb40 tx=0x7fe864005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.294+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe86c0968b0 con 0x7fe87c107600 2024-03-11T19:53:16.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:16.040+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:16.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:16.040+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:16.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:16.040+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:16.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:16.040+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:16.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:16.040+0000 7f0a1d000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:16.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.548+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe87c112fe0 con 0x7fe87c107600 2024-03-11T19:53:16.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.563+0000 7fe873e00640 1 -- 172.21.15.107:0/1386129622 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe86c099f70 con 0x7fe87c107600 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.565+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 msgr2=0x7fe8600a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.565+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 0x7fe8600a3ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe87c19eb40 tx=0x7fe864005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.565+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 msgr2=0x7fe87c19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.565+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00e9f0 tx=0x7fe86c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 shutdown_connections 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fe8600a1a20 0x7fe8600a3ee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe87c1091b0 0x7fe87c1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe87c1087f0 0x7fe87c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 --2- 172.21.15.107:0/1386129622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe87c107600 0x7fe87c19d630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 >> 172.21.15.107:0/1386129622 conn(0x7fe87c0765c0 msgr2=0x7fe87c10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 shutdown_connections 2024-03-11T19:53:16.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.566+0000 7fe880a00640 1 -- 172.21.15.107:0/1386129622 wait complete. 2024-03-11T19:53:16.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-11T19:53:16.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 -- 172.21.15.107:0/2791722398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89380751c0 msgr2=0x7f8938073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 --2- 172.21.15.107:0/2791722398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89380751c0 0x7f8938073610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f892c009ca0 tx=0x7f892c02c100 comp rx=0 tx=0).stop 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 -- 172.21.15.107:0/2791722398 shutdown_connections 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 --2- 172.21.15.107:0/2791722398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f893810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 --2- 172.21.15.107:0/2791722398 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8938073be0 0x7f8938074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 --2- 172.21.15.107:0/2791722398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89380751c0 0x7f8938073610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 -- 172.21.15.107:0/2791722398 >> 172.21.15.107:0/2791722398 conn(0x7f89380fc740 msgr2=0x7f89380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:16.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 -- 172.21.15.107:0/2791722398 shutdown_connections 2024-03-11T19:53:16.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.769+0000 7f8940000640 1 -- 172.21.15.107:0/2791722398 wait complete. 2024-03-11T19:53:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 Processor -- start 2024-03-11T19:53:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 -- start start 2024-03-11T19:53:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 0x7f8938199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f89381a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381a0e70 con 0x7f8938073be0 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381a0fe0 con 0x7f8938109070 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.770+0000 7f8940000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381a11a0 con 0x7f89380751c0 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 0x7f8938199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49720/0 (socket says 172.21.15.107:49720) 2024-03-11T19:53:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 -- 172.21.15.107:0/1997579913 learned_addr learned my addr 172.21.15.107:0/1997579913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893e000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f89381a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 msgr2=0x7f89381a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f89381a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 msgr2=0x7f8938199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 0x7f8938199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 -- 172.21.15.107:0/1997579913 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c009900 con 0x7f89380751c0 2024-03-11T19:53:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893e000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f89381a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:16.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893d600640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 0x7f8938199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:16.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.771+0000 7f893cc00640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f892400e9f0 tx=0x7f892400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:16.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.772+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924009720 con 0x7f89380751c0 2024-03-11T19:53:16.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.772+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892400cd80 con 0x7f89380751c0 2024-03-11T19:53:16.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.772+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89381a1480 con 0x7f89380751c0 2024-03-11T19:53:16.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.772+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924010690 con 0x7f89380751c0 2024-03-11T19:53:16.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.772+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89381a1990 con 0x7f89380751c0 2024-03-11T19:53:16.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.773+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8938107da0 con 0x7f89380751c0 2024-03-11T19:53:16.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.776+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) v1 ==== 136297+0+0 (secure 0 0 0) 0x7f89240046c0 con 0x7f89380751c0 2024-03-11T19:53:16.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.777+0000 7f8936000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 0x7f89200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.777+0000 7f893d600640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 msgr2=0x7f89200a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.777+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8924014070 con 0x7f89380751c0 2024-03-11T19:53:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.777+0000 7f893d600640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 0x7f89200a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:53:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.778+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8924096950 con 0x7f89380751c0 2024-03-11T19:53:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.976+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f893810edb0 con 0x7f89380751c0 2024-03-11T19:53:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.978+0000 7f893d600640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 msgr2=0x7f89200a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.978+0000 7f893d600640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 0x7f89200a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:53:16.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.982+0000 7f8936000640 1 -- 172.21.15.107:0/1997579913 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7f892409a010 con 0x7f89380751c0 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 msgr2=0x7f89200a4060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 0x7f89200a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 msgr2=0x7f89381998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f892400e9f0 tx=0x7f892400edf0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 shutdown_connections 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f89200a1ba0 0x7f89200a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8938109070 0x7f89381a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380751c0 0x7f89381998c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 --2- 172.21.15.107:0/1997579913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8938073be0 0x7f8938199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:16.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 >> 172.21.15.107:0/1997579913 conn(0x7f89380fc740 msgr2=0x7f8938104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:16.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 shutdown_connections 2024-03-11T19:53:16.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:16.985+0000 7f8940000640 1 -- 172.21.15.107:0/1997579913 wait complete. 2024-03-11T19:53:17.005 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setuser ceph since I am not root 2024-03-11T19:53:17.005 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: ignoring --setgroup ceph since I am not root 2024-03-11T19:53:17.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:53:17.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:53:17.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-mon[31712]: from='client.25415 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:17.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-mon[31712]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:53:17.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:16 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:53:17.008 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mon[36397]: from='client.25415 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mon[36397]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: ignoring --setuser ceph since I am not root 2024-03-11T19:53:17.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: ignoring --setgroup ceph since I am not root 2024-03-11T19:53:17.014 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:16 smithi122 ceph-mgr[37788]: -- 172.21.15.122:0/3429138040 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5606411301a0 con 0x560641106c00 2024-03-11T19:53:17.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: ignoring --setuser ceph since I am not root 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: ignoring --setgroup ceph since I am not root 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mgr[38607]: -- 172.21.15.145:0/510818966 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648a6e981a0 con 0x5648a6e6e800 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mon[36730]: from='client.25415 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mon[36730]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:53:17.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:16 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 -- 172.21.15.107:0/1024816446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0075b40 msgr2=0x7fadd0075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 --2- 172.21.15.107:0/1024816446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0075b40 0x7fadd0075f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009900 tx=0x7fadc402bc40 comp rx=0 tx=0).stop 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 -- 172.21.15.107:0/1024816446 shutdown_connections 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 --2- 172.21.15.107:0/1024816446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0077720 0x7fadd0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 --2- 172.21.15.107:0/1024816446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0076d80 0x7fadd00771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 --2- 172.21.15.107:0/1024816446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0075b40 0x7fadd0075f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.203+0000 7fadd8200640 1 -- 172.21.15.107:0/1024816446 >> 172.21.15.107:0/1024816446 conn(0x7fadd00fe970 msgr2=0x7fadd0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.204+0000 7fadd8200640 1 -- 172.21.15.107:0/1024816446 shutdown_connections 2024-03-11T19:53:17.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.204+0000 7fadd8200640 1 -- 172.21.15.107:0/1024816446 wait complete. 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 Processor -- start 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 -- start start 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0076d80 0x7fadd019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0077720 0x7fadd01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a52a0 con 0x7fadd0076d80 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a5410 con 0x7fadd0075b40 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.205+0000 7fadd8200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01a55d0 con 0x7fadd0077720 2024-03-11T19:53:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43928/0 (socket says 172.21.15.107:43928) 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 -- 172.21.15.107:0/3444097563 learned_addr learned my addr 172.21.15.107:0/3444097563 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd6200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0077720 0x7fadd01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd4e00640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0076d80 0x7fadd019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0077720 msgr2=0x7fadd01a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0077720 0x7fadd01a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0076d80 msgr2=0x7fadd019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0076d80 0x7fadd019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 -- 172.21.15.107:0/3444097563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4009560 con 0x7fadd0075b40 2024-03-11T19:53:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.206+0000 7fadd5800640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006010 tx=0x7fadc4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.207+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc40043d0 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.207+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc40052d0 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.207+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd01a5850 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.207+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4003cc0 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.207+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd01a5db0 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.208+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd010fd50 con 0x7fadd0075b40 2024-03-11T19:53:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.210+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) v1 ==== 136297+0+0 (secure 0 0 0) 0x7fadc4005440 con 0x7fadd0075b40 2024-03-11T19:53:17.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.211+0000 7fadce000640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 0x7fadb80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.211+0000 7fadd4e00640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 msgr2=0x7fadb80a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:17.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.212+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fadc40efc80 con 0x7fadd0075b40 2024-03-11T19:53:17.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.212+0000 7fadd4e00640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 0x7fadb80a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:53:17.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.213+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadc40b8a80 con 0x7fadd0075b40 2024-03-11T19:53:17.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.410+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fadd01131e0 con 0x7fadd0075b40 2024-03-11T19:53:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.413+0000 7fadd4e00640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 msgr2=0x7fadb80a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:17.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.413+0000 7fadd4e00640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 0x7fadb80a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:53:17.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.417+0000 7fadce000640 1 -- 172.21.15.107:0/3444097563 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) v1 ==== 133+0+0 (secure 0 0 0) 0x7fadc40bc140 con 0x7fadd0075b40 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 msgr2=0x7fadb80a4060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 0x7fadb80a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 msgr2=0x7fadd019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006010 tx=0x7fadc4005e80 comp rx=0 tx=0).stop 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 shutdown_connections 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7fadb80a1ba0 0x7fadb80a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fadd0077720 0x7fadd01a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadd0076d80 0x7fadd019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 --2- 172.21.15.107:0/3444097563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadd0075b40 0x7fadd019d800 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 >> 172.21.15.107:0/3444097563 conn(0x7fadd00fe970 msgr2=0x7fadd0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 shutdown_connections 2024-03-11T19:53:17.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.420+0000 7fadd8200640 1 -- 172.21.15.107:0/3444097563 wait complete. 2024-03-11T19:53:17.443 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-11T19:53:17.513 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:17.132+0000 7fbeccf96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:53:17.513 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:17.227+0000 7fbeccf96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:53:17.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:17.137+0000 7f36b5748200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:53:17.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:17.229+0000 7f36b5748200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:53:17.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 -- 172.21.15.107:0/3509829907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 msgr2=0x7f7c2c075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 --2- 172.21.15.107:0/3509829907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c075f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009860 tx=0x7f7c1402bb70 comp rx=0 tx=0).stop 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 -- 172.21.15.107:0/3509829907 shutdown_connections 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 --2- 172.21.15.107:0/3509829907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c2c077720 0x7f7c2c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 --2- 172.21.15.107:0/3509829907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c076d80 0x7f7c2c0771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 --2- 172.21.15.107:0/3509829907 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c075f60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 -- 172.21.15.107:0/3509829907 >> 172.21.15.107:0/3509829907 conn(0x7f7c2c0fe970 msgr2=0x7f7c2c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 -- 172.21.15.107:0/3509829907 shutdown_connections 2024-03-11T19:53:17.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.638+0000 7f7c31c00640 1 -- 172.21.15.107:0/3509829907 wait complete. 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.639+0000 7f7c31c00640 1 Processor -- start 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 -- start start 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c2c076d80 0x7f7c2c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c077720 0x7f7c2c1a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c1a5260 con 0x7f7c2c077720 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c1a53d0 con 0x7f7c2c076d80 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c31c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c1a5590 con 0x7f7c2c075b40 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c2aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c2aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:49746/0 (socket says 172.21.15.107:49746) 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c2aa00640 1 -- 172.21.15.107:0/2893717596 learned_addr learned my addr 172.21.15.107:0/2893717596 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c2b400640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c077720 0x7f7c2c1a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.640+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c2c076d80 0x7f7c2c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c2c076d80 msgr2=0x7f7c2c19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c2c076d80 0x7f7c2c19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c077720 msgr2=0x7f7c2c1a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c077720 0x7f7c2c1a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009560 con 0x7f7c2c075b40 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2b400640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c2c077720 0x7f7c2c1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c2aa00640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c19d770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c14005940 tx=0x7f7c140045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:17.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1402c510 con 0x7f7c2c075b40 2024-03-11T19:53:17.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c1402cb50 con 0x7f7c2c075b40 2024-03-11T19:53:17.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14032860 con 0x7f7c2c075b40 2024-03-11T19:53:17.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c14009490 con 0x7f7c2c075b40 2024-03-11T19:53:17.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.641+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c1a5a40 con 0x7f7c2c075b40 2024-03-11T19:53:17.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.643+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c10fd50 con 0x7f7c2c075b40 2024-03-11T19:53:17.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.644+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 19) v1 ==== 136297+0+0 (secure 0 0 0) 0x7f7c1403b040 con 0x7f7c2c075b40 2024-03-11T19:53:17.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.646+0000 7f7c23400640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.646+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:17.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.646+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7c140ef500 con 0x7f7c2c075b40 2024-03-11T19:53:17.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.646+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:53:17.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.649+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c140b8300 con 0x7f7c2c075b40 2024-03-11T19:53:17.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:17.377+0000 7f9bde103200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:53:17.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:17.502+0000 7f9bde103200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:53:17.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:53:17.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-mon[31712]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:53:17.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' 2024-03-11T19:53:17.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:17 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' 2024-03-11T19:53:17.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.847+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:17.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.847+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:53:17.872 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:53:17.872 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-mon[36397]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:53:17.872 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' 2024-03-11T19:53:17.872 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' 2024-03-11T19:53:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:17.868+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c2c02ce40 con 0x7f7c100a1ba0 2024-03-11T19:53:17.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1386129622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-11T19:53:17.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-mon[36730]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-03-11T19:53:17.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' 2024-03-11T19:53:17.875 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' 2024-03-11T19:53:18.202 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:17.867+0000 7fbeccf96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:53:18.203 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:17.960+0000 7fbeccf96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:53:18.203 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.102+0000 7fbeccf96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:53:18.203 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.198+0000 7fbeccf96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:53:18.206 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:17.870+0000 7f36b5748200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:53:18.206 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:17.964+0000 7f36b5748200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:53:18.206 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.106+0000 7f36b5748200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:53:18.206 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.202+0000 7f36b5748200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:53:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:18.248+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:18.248+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:53:18.512 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.281+0000 7fbeccf96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:53:18.537 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.284+0000 7f36b5748200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:53:18.596 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:18.295+0000 7f9bde103200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:53:18.596 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:18.415+0000 7f9bde103200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:53:18.838 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.543+0000 7fbeccf96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:53:18.838 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.653+0000 7fbeccf96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:18.838 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.739+0000 7fbeccf96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:53:18.838 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:18.833+0000 7fbeccf96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:53:18.842 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.548+0000 7f36b5748200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:53:18.842 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.657+0000 7f36b5748200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:18.842 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.743+0000 7f36b5748200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:53:18.842 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:18.837+0000 7f36b5748200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:53:19.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:18.590+0000 7f9bde103200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:53:19.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:18.715+0000 7f9bde103200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:53:19.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:18.824+0000 7f9bde103200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:19.049+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:19.049+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:53:19.263 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:19.008+0000 7fbeccf96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:53:19.287 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:19.014+0000 7f36b5748200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:53:19.530 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:19.153+0000 7f9bde103200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:53:19.531 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:19.291+0000 7f9bde103200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:19.531 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:19.401+0000 7f9bde103200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:53:19.794 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:19.525+0000 7f9bde103200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:53:19.795 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:19.743+0000 7f9bde103200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:53:20.263 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:19.968+0000 7fbeccf96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:53:20.263 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:20.097+0000 7fbeccf96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:53:20.287 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:19.974+0000 7f36b5748200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:53:20.287 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:20.105+0000 7f36b5748200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:53:20.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:20.651+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:20.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:20.651+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:53:20.716 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:20.428+0000 7fbeccf96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:53:20.716 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:20.519+0000 7fbeccf96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:53:20.734 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:20.440+0000 7f36b5748200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:53:20.734 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:20.532+0000 7f36b5748200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:53:21.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:20.711+0000 7fbeccf96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:21.013 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:20.893+0000 7fbeccf96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:53:21.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:20.729+0000 7f36b5748200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:21.037 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:20.913+0000 7f36b5748200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:53:21.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:21.009+0000 7f9bde103200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:53:21.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:21.180+0000 7f9bde103200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:53:21.436 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:21.174+0000 7fbeccf96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:21.436 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:21.431+0000 7fbeccf96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:21.463 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:21.196+0000 7f36b5748200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:21.764 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:21.664+0000 7fbeccf96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:53:21.787 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:21.459+0000 7f36b5748200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:21.787 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:21.692+0000 7f36b5748200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:53:21.979 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:21.597+0000 7f9bde103200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:53:21.979 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:21.716+0000 7f9bde103200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:53:21.979 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:21.973+0000 7f9bde103200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:22.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:22.215+0000 7f9bde103200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:53:22.763 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:22.453+0000 7fbeccf96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:53:22.787 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:22.486+0000 7f36b5748200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:53:22.921 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:22.591+0000 7f9bde103200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:53:23.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:22.915+0000 7f9bde103200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:53:23.337 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:23.020+0000 7fbeccf96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:53:23.337 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:23.117+0000 7fbeccf96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:53:23.337 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y[37784]: 2024-03-11T19:53:23.332+0000 7fbeccf96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:53:23.379 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:23.061+0000 7f36b5748200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:53:23.379 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:23.159+0000 7f36b5748200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:53:23.379 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z[38603]: 2024-03-11T19:53:23.374+0000 7f36b5748200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:53:23.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:23.418+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mgrmap(e 20) v1 ==== 136297+0+0 (secure 0 0 0) 0x7f7c140b86d0 con 0x7f7c2c075b40 2024-03-11T19:53:23.544 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:23.229+0000 7f9bde103200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: Standby manager daemon y restarted 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: Standby manager daemon y started 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: Standby manager daemon z restarted 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: Standby manager daemon z started 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:53:23.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:23 smithi107 ceph-mon[31712]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: Standby manager daemon y restarted 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: Standby manager daemon y started 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: Standby manager daemon z restarted 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: Standby manager daemon z started 2024-03-11T19:53:23.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:53:23.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:53:23.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:23 smithi122 ceph-mon[36397]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: Standby manager daemon y restarted 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: Standby manager daemon y started 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.122:0/593151067' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: Standby manager daemon z restarted 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: Standby manager daemon z started 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:53:23.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:23 smithi145 ceph-mon[36730]: from='mgr.? 172.21.15.145:0/3456884135' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:53:23.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:23.855+0000 7f7c2a000640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1546798510 2024-03-11T19:53:23.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:23.855+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-11T19:53:24.544 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:24 smithi107 ceph-mon[31712]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-03-11T19:53:24.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:24.264+0000 7f9bde103200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:53:24.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:24 smithi122 ceph-mon[36397]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-03-11T19:53:24.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:24 smithi145 ceph-mon[36730]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-03-11T19:53:25.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:24.964+0000 7f9bde103200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:53:25.294 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.096+0000 7f9bde103200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:53:25.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:25.405+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 8 ==== mgrmap(e 21) v1 ==== 135964+0+0 (secure 0 0 0) 0x7f7c140ecc70 con 0x7f7c2c075b40 2024-03-11T19:53:25.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:25.406+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 msgr2=0x7f7c100a4060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:53:25.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:25.406+0000 7f7c23400640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/1546798510,v1:172.21.15.107:6801/1546798510] conn(0x7f7c100a1ba0 0x7f7c100a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: Active manager daemon x restarted 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: Activating manager daemon x 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: mgrmap e21: x(active, starting, since 0.0163749s), standbys: y, z 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:53:25.695 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:53:25.696 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:53:25.697 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:25 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.380+0000 7f9bde103200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.692+0000 7f9b33c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.692+0000 7f9b33c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.692+0000 7f9b33c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.692+0000 7f9b33c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:25.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.692+0000 7f9b33c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: Active manager daemon x restarted 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: Activating manager daemon x 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: mgrmap e21: x(active, starting, since 0.0163749s), standbys: y, z 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:53:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:53:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:53:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:53:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: Active manager daemon x restarted 2024-03-11T19:53:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: Activating manager daemon x 2024-03-11T19:53:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: osdmap e96: 12 total, 12 up, 12 in 2024-03-11T19:53:25.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: mgrmap e21: x(active, starting, since 0.0163749s), standbys: y, z 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:53:25.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:53:25.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:53:25.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:53:25.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:53:25.766 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:25 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:53:26.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.693+0000 7f9b30000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:26.044 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.693+0000 7f9b30000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:26.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.693+0000 7f9b30000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:26.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.693+0000 7f9b30000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:26.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:53:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:53:25.693+0000 7f9b30000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:53:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.412+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mon.1 v2:172.21.15.145:3300/0 9 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f7c14003780 con 0x7f7c2c075b40 2024-03-11T19:53:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.413+0000 7f7c23400640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f7c10034d00 0x7f7c100350e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.413+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c2c02ce40 con 0x7f7c10034d00 2024-03-11T19:53:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.422+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f7c10034d00 0x7f7c100350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:26.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.435+0000 7f7c2a000640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f7c10034d00 0x7f7c100350e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c19ec80 tx=0x7f7c1c009260 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:26.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.520+0000 7f7c23400640 1 -- 172.21.15.107:0/2893717596 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7c2c02ce40 con 0x7f7c10034d00 2024-03-11T19:53:26.525 INFO:teuthology.orchestra.run.smithi107.stdout:Schedule set for path / 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f7c10034d00 msgr2=0x7f7c100350e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f7c10034d00 0x7f7c100350e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c19ec80 tx=0x7f7c1c009260 comp rx=0 tx=0).stop 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 msgr2=0x7f7c2c19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 --2- 172.21.15.107:0/2893717596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7c2c075b40 0x7f7c2c19d770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c14005940 tx=0x7f7c140045e0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c2b400640 1 -- 172.21.15.107:0/2893717596 reap_dead start 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 shutdown_connections 2024-03-11T19:53:26.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 >> 172.21.15.107:0/2893717596 conn(0x7f7c2c0fe970 msgr2=0x7f7c2c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:26.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.522+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 shutdown_connections 2024-03-11T19:53:26.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.523+0000 7f7c31c00640 1 -- 172.21.15.107:0/2893717596 wait complete. 2024-03-11T19:53:26.547 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: Manager daemon x is now available 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.712 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:26 smithi122 ceph-mon[36397]: mgrmap e22: x(active, since 1.03423s), standbys: y, z 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 -- 172.21.15.107:0/2518940949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1045d0 msgr2=0x7f61ac104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 --2- 172.21.15.107:0/2518940949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1045d0 0x7f61ac104a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6198009900 tx=0x7f619802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 -- 172.21.15.107:0/2518940949 shutdown_connections 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 --2- 172.21.15.107:0/2518940949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 0x7f61ac10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 --2- 172.21.15.107:0/2518940949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1045d0 0x7f61ac104a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 --2- 172.21.15.107:0/2518940949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1033e0 0x7f61ac103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 -- 172.21.15.107:0/2518940949 >> 172.21.15.107:0/2518940949 conn(0x7f61ac0fe970 msgr2=0x7f61ac100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 -- 172.21.15.107:0/2518940949 shutdown_connections 2024-03-11T19:53:26.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.721+0000 7f61b2600640 1 -- 172.21.15.107:0/2518940949 wait complete. 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 Processor -- start 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 -- start start 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1045d0 0x7f61ac1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 0x7f61ac1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1a0ea0 con 0x7f61ac1033e0 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1a1010 con 0x7f61ac1045d0 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61b2600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1a11d0 con 0x7f61ac104f90 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56756/0 (socket says 172.21.15.107:56756) 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61ab400640 1 -- 172.21.15.107:0/2157826214 learned_addr learned my addr 172.21.15.107:0/2157826214 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61aaa00640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1045d0 0x7f61ac1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.722+0000 7f61abe00640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 0x7f61ac1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:26.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 -- 172.21.15.107:0/2157826214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 msgr2=0x7f61ac1a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 0x7f61ac1a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 -- 172.21.15.107:0/2157826214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1045d0 msgr2=0x7f61ac1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1045d0 0x7f61ac1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 -- 172.21.15.107:0/2157826214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c009490 con 0x7f61ac1033e0 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61ab400640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9c0 tx=0x7f619c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:26.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cde0 con 0x7f61ac1033e0 2024-03-11T19:53:26.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.723+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6198009560 con 0x7f61ac1033e0 2024-03-11T19:53:26.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.724+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac1a1850 con 0x7f61ac1033e0 2024-03-11T19:53:26.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.724+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f619c010910 con 0x7f61ac1033e0 2024-03-11T19:53:26.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.724+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c018980 con 0x7f61ac1033e0 2024-03-11T19:53:26.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.725+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c005290 con 0x7f61ac1033e0 2024-03-11T19:53:26.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.727+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f619c010a80 con 0x7f61ac1033e0 2024-03-11T19:53:26.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.728+0000 7f61a3e00640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 0x7f61900a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:26.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.729+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f619c014070 con 0x7f61ac1033e0 2024-03-11T19:53:26.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.729+0000 7f61aaa00640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 0x7f61900a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:26.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.730+0000 7f61aaa00640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 0x7f61900a3e20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6198000c00 tx=0x7f619802c7e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:26.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.733+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f619c096a20 con 0x7f61ac1033e0 2024-03-11T19:53:26.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: Manager daemon x is now available 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:26 smithi145 ceph-mon[36730]: mgrmap e22: x(active, since 1.03423s), standbys: y, z 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: Manager daemon x is now available 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:53:26.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:26 smithi107 ceph-mon[31712]: mgrmap e22: x(active, since 1.03423s), standbys: y, z 2024-03-11T19:53:26.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:26.959+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f618c000c60 con 0x7f61900a1960 2024-03-11T19:53:27.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.078+0000 7f61a3e00640 1 -- 172.21.15.107:0/2157826214 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f618c000c60 con 0x7f61900a1960 2024-03-11T19:53:27.083 INFO:teuthology.orchestra.run.smithi107.stdout:Retention added to path / 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 msgr2=0x7f61900a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 0x7f61900a3e20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6198000c00 tx=0x7f619802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 msgr2=0x7f61ac1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9c0 tx=0x7f619c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 shutdown_connections 2024-03-11T19:53:27.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f61900a1960 0x7f61900a3e20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61ac1045d0 0x7f61ac1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac104f90 0x7f61ac1a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 --2- 172.21.15.107:0/2157826214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac1033e0 0x7f61ac1993b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 >> 172.21.15.107:0/2157826214 conn(0x7f61ac0fe970 msgr2=0x7f61ac107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.082+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 shutdown_connections 2024-03-11T19:53:27.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.083+0000 7f61b2600640 1 -- 172.21.15.107:0/2157826214 wait complete. 2024-03-11T19:53:27.104 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-11T19:53:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.276+0000 7f8576600640 1 -- 172.21.15.107:0/3260082162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701033e0 msgr2=0x7f8570103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.276+0000 7f8576600640 1 --2- 172.21.15.107:0/3260082162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701033e0 0x7f8570103800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8560009900 tx=0x7f856002bc40 comp rx=0 tx=0).stop 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 -- 172.21.15.107:0/3260082162 shutdown_connections 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 --2- 172.21.15.107:0/3260082162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8570104f90 0x7f857010b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 --2- 172.21.15.107:0/3260082162 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85701045d0 0x7f8570104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 --2- 172.21.15.107:0/3260082162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701033e0 0x7f8570103800 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 -- 172.21.15.107:0/3260082162 >> 172.21.15.107:0/3260082162 conn(0x7f85700fe970 msgr2=0x7f8570100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 -- 172.21.15.107:0/3260082162 shutdown_connections 2024-03-11T19:53:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.277+0000 7f8576600640 1 -- 172.21.15.107:0/3260082162 wait complete. 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 Processor -- start 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 -- start start 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701045d0 0x7f8570199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8570104f90 0x7f85701a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701a0eb0 con 0x7f85701033e0 2024-03-11T19:53:27.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701a1020 con 0x7f85701045d0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8576600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701a11e0 con 0x7f8570104f90 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f856f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f856f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56770/0 (socket says 172.21.15.107:56770) 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f8566a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701045d0 0x7f8570199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f856f400640 1 -- 172.21.15.107:0/1764836428 learned_addr learned my addr 172.21.15.107:0/1764836428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.278+0000 7f856fe00640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8570104f90 0x7f85701a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 -- 172.21.15.107:0/1764836428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8570104f90 msgr2=0x7f85701a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8570104f90 0x7f85701a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 -- 172.21.15.107:0/1764836428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701045d0 msgr2=0x7f8570199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701045d0 0x7f8570199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 -- 172.21.15.107:0/1764836428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009560 con 0x7f85701033e0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856f400640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8560005ab0 tx=0x7f8560005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600043d0 con 0x7f85701033e0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85600052d0 con 0x7f85701033e0 2024-03-11T19:53:27.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85701a1460 con 0x7f85701033e0 2024-03-11T19:53:27.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.279+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560003d40 con 0x7f85701033e0 2024-03-11T19:53:27.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.280+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85701a1970 con 0x7f85701033e0 2024-03-11T19:53:27.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.281+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857010a6a0 con 0x7f85701033e0 2024-03-11T19:53:27.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.282+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f8560005440 con 0x7f85701033e0 2024-03-11T19:53:27.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.283+0000 7f856cc00640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 0x7f85540a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.284+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f85600ef640 con 0x7f85701033e0 2024-03-11T19:53:27.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.284+0000 7f8566a00640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 0x7f85540a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.284+0000 7f8566a00640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 0x7f85540a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f857019a8d0 tx=0x7f8558005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:27.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.287+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85600b8620 con 0x7f85701033e0 2024-03-11T19:53:27.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.470+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 23) v1 ==== 136400+0+0 (secure 0 0 0) 0x7f85600b89f0 con 0x7f85701033e0 2024-03-11T19:53:27.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.511+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f857002ce40 con 0x7f85540a18e0 2024-03-11T19:53:27.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.521+0000 7f856cc00640 1 -- 172.21.15.107:0/1764836428 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f857002ce40 con 0x7f85540a18e0 2024-03-11T19:53:27.525 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-11T00:00:00", "created": "2024-03-11T19:53:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 msgr2=0x7f85540a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 0x7f85540a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f857019a8d0 tx=0x7f8558005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 msgr2=0x7f85701993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8560005ab0 tx=0x7f8560005e50 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 shutdown_connections 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f85540a18e0 0x7f85540a3da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85701045d0 0x7f8570199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8570104f90 0x7f85701a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 --2- 172.21.15.107:0/1764836428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85701033e0 0x7f85701993c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 >> 172.21.15.107:0/1764836428 conn(0x7f85700fe970 msgr2=0x7f8570107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:27.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 shutdown_connections 2024-03-11T19:53:27.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.523+0000 7f8576600640 1 -- 172.21.15.107:0/1764836428 wait complete. 2024-03-11T19:53:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: [11/Mar/2024:19:53:25] ENGINE Bus STARTING 2024-03-11T19:53:27.537 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: [11/Mar/2024:19:53:26] ENGINE Serving on http://172.21.15.107:8765 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: [11/Mar/2024:19:53:26] ENGINE Serving on https://172.21.15.107:7150 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: [11/Mar/2024:19:53:26] ENGINE Bus STARTED 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.538 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:27 smithi145 ceph-mon[36730]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:27.547 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-11T19:53:27.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.747+0000 7fb09b800640 1 -- 172.21.15.107:0/1578961131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0941033e0 msgr2=0x7fb094103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.747+0000 7fb09b800640 1 --2- 172.21.15.107:0/1578961131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0941033e0 0x7fb094103800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 -- 172.21.15.107:0/1578961131 shutdown_connections 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 --2- 172.21.15.107:0/1578961131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb094104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 --2- 172.21.15.107:0/1578961131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb094104f90 0x7fb09410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 --2- 172.21.15.107:0/1578961131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0941033e0 0x7fb094103800 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 -- 172.21.15.107:0/1578961131 >> 172.21.15.107:0/1578961131 conn(0x7fb0940fe970 msgr2=0x7fb094100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 -- 172.21.15.107:0/1578961131 shutdown_connections 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.748+0000 7fb09b800640 1 -- 172.21.15.107:0/1578961131 wait complete. 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.749+0000 7fb09b800640 1 Processor -- start 2024-03-11T19:53:27.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.749+0000 7fb09b800640 1 -- start start 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.749+0000 7fb09b800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.749+0000 7fb09b800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.749+0000 7fb09b800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 0x7fb0941a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb09b800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941a0ea0 con 0x7fb0941033e0 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb09b800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941a1010 con 0x7fb0941045d0 2024-03-11T19:53:27.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb09b800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941a11d0 con 0x7fb094104f90 2024-03-11T19:53:27.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb098e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb098e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56800/0 (socket says 172.21.15.107:56800) 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb099800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 0x7fb0941a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb098e00640 1 -- 172.21.15.107:0/3364324873 learned_addr learned my addr 172.21.15.107:0/3364324873 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39740/0 (socket says 172.21.15.107:39740) 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 -- 172.21.15.107:0/3364324873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 msgr2=0x7fb0941a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 0x7fb0941a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 -- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 msgr2=0x7fb0941993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb093e00640 1 -- 172.21.15.107:0/3364324873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088009560 con 0x7fb0941045d0 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb098e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.750+0000 7fb099800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 0x7fb0941a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb093e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb08000e9f0 tx=0x7fb08000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08000ccc0 con 0x7fb0941045d0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb08000ce20 con 0x7fb0941045d0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0941a14b0 con 0x7fb0941045d0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080018920 con 0x7fb0941045d0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.751+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0941a19c0 con 0x7fb0941045d0 2024-03-11T19:53:27.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.752+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09410a6a0 con 0x7fb0941045d0 2024-03-11T19:53:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.754+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) v1 ==== 136400+0+0 (secure 0 0 0) 0x7fb080010430 con 0x7fb0941045d0 2024-03-11T19:53:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.757+0000 7fb091600640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 0x7fb07c0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.757+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb080014070 con 0x7fb0941045d0 2024-03-11T19:53:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.757+0000 7fb098e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 0x7fb07c0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:27.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.758+0000 7fb098e00640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 0x7fb07c0a4100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb088006010 tx=0x7fb08802c7e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: [11/Mar/2024:19:53:25] ENGINE Bus STARTING 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: [11/Mar/2024:19:53:26] ENGINE Serving on http://172.21.15.107:8765 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: [11/Mar/2024:19:53:26] ENGINE Serving on https://172.21.15.107:7150 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: [11/Mar/2024:19:53:26] ENGINE Bus STARTED 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:27 smithi122 ceph-mon[36397]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:27.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.760+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb080052d20 con 0x7fb0941045d0 2024-03-11T19:53:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: [11/Mar/2024:19:53:25] ENGINE Bus STARTING 2024-03-11T19:53:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: [11/Mar/2024:19:53:26] ENGINE Serving on http://172.21.15.107:8765 2024-03-11T19:53:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: [11/Mar/2024:19:53:26] ENGINE Serving on https://172.21.15.107:7150 2024-03-11T19:53:27.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: [11/Mar/2024:19:53:26] ENGINE Bus STARTED 2024-03-11T19:53:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: from='client.25468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:27.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:27 smithi107 ceph-mon[31712]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:27.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.987+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb09402ce10 con 0x7fb07c0a1c40 2024-03-11T19:53:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:27.997+0000 7fb091600640 1 -- 172.21.15.107:0/3364324873 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fb09402ce10 con 0x7fb07c0a1c40 2024-03-11T19:53:28.002 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 msgr2=0x7fb07c0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 0x7fb07c0a4100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb088006010 tx=0x7fb08802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 msgr2=0x7fb0941998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb08000e9f0 tx=0x7fb08000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 shutdown_connections 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb07c0a1c40 0x7fb07c0a4100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0941045d0 0x7fb0941998f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb094104f90 0x7fb0941a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 --2- 172.21.15.107:0/3364324873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0941033e0 0x7fb0941993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:28.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 >> 172.21.15.107:0/3364324873 conn(0x7fb0940fe970 msgr2=0x7fb094107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:28.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 shutdown_connections 2024-03-11T19:53:28.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T19:53:28.000+0000 7fb09b800640 1 -- 172.21.15.107:0/3364324873 wait complete. 2024-03-11T19:53:28.024 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-11T19:53:28.102 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-11T19:53:28.113 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-11T19:53:28.136 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:53:28.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-11T19:53:28.142 INFO:teuthology.run_tasks:Running task workunit... 2024-03-11T19:53:28.154 INFO:tasks.workunit:Pulling workunits from ref b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T19:53:28.155 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-11T19:53:28.155 INFO:tasks.workunit:timeout=6h 2024-03-11T19:53:28.155 INFO:tasks.workunit:cleanup=True 2024-03-11T19:53:28.155 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:28.199 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-11 19:50:31.304462398 +0000 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-11 19:50:31.304462398 +0000 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-11 19:53:14.208647037 +0000 2024-03-11T19:53:28.200 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-11 19:50:31.304462398 +0000 2024-03-11T19:53:28.200 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-11T19:53:28.200 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-11T19:53:28.320 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T19:53:28.362 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:28 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:28.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-11T19:53:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:28 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:28.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:28 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='client.25517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T19:53:29.764 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.765 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:29 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='client.25517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.789 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:29 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='client.25517 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:53:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:29 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:30 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T19:53:30.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:30 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:30.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:30.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:30 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:53:31.763 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:31 smithi122 ceph-mon[36397]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:53:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:53:31.788 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:31 smithi145 ceph-mon[36730]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: Updating smithi122:/etc/ceph/ceph.conf 2024-03-11T19:53:31.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:31 smithi107 ceph-mon[31712]: Updating smithi145:/etc/ceph/ceph.conf 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:32 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:32 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi145:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: Updating smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:53:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:33.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:32 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:33 smithi122 ceph-mon[36397]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:33 smithi122 ceph-mon[36397]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:33 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:33 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:33 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:33 smithi145 ceph-mon[36730]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:33 smithi145 ceph-mon[36730]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:33 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:33 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:33 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:33 smithi107 ceph-mon[31712]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:53:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:33 smithi107 ceph-mon[31712]: Updating smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:53:34.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:33 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:33 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:34.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:33 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:34 smithi122 ceph-mon[36397]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-11T19:53:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:34 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:34 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:34 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:34 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:34 smithi145 ceph-mon[36730]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-11T19:53:35.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:34 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:35.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:34 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:35.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:34 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:35.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:34 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:35.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:34 smithi107 ceph-mon[31712]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-11T19:53:35.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:34 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:35.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:34 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:35.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:34 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:35.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:34 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:53:37.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:36 smithi122 ceph-mon[36397]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-11T19:53:37.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:36 smithi145 ceph-mon[36730]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-11T19:53:37.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:36 smithi107 ceph-mon[31712]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-11T19:53:38.840 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:38 smithi145 ceph-mon[36730]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-11T19:53:39.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:38 smithi122 ceph-mon[36397]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-11T19:53:39.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:38 smithi107 ceph-mon[31712]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-11T19:53:41.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:40 smithi122 ceph-mon[36397]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:41.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:40 smithi145 ceph-mon[36730]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:41.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:40 smithi107 ceph-mon[31712]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 -- 172.21.15.107:0/2630406462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc073e40 msgr2=0x7f93cc111a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 --2- 172.21.15.107:0/2630406462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc073e40 0x7f93cc111a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b402bcd0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 -- 172.21.15.107:0/2630406462 shutdown_connections 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 --2- 172.21.15.107:0/2630406462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc111f70 0x7f93cc114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 --2- 172.21.15.107:0/2630406462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc073e40 0x7f93cc111a30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 --2- 172.21.15.107:0/2630406462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073520 0x7f93cc073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 -- 172.21.15.107:0/2630406462 >> 172.21.15.107:0/2630406462 conn(0x7f93cc0fca90 msgr2=0x7f93cc0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 -- 172.21.15.107:0/2630406462 shutdown_connections 2024-03-11T19:53:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.381+0000 7f93d0c00640 1 -- 172.21.15.107:0/2630406462 wait complete. 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 Processor -- start 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 -- start start 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073e40 0x7f93cc1a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc111f70 0x7f93cc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc19e110 con 0x7f93cc111f70 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc19e280 con 0x7f93cc073e40 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.382+0000 7f93d0c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc19e440 con 0x7f93cc073520 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43358/0 (socket says 172.21.15.107:43358) 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 -- 172.21.15.107:0/647439303 learned_addr learned my addr 172.21.15.107:0/647439303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93c3e00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073e40 0x7f93cc1a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cbe00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc111f70 0x7f93cc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 -- 172.21.15.107:0/647439303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073e40 msgr2=0x7f93cc1a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073e40 0x7f93cc1a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 -- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc111f70 msgr2=0x7f93cc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc111f70 0x7f93cc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 -- 172.21.15.107:0/647439303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009560 con 0x7f93cc073520 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.383+0000 7f93cb400640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e9f0 tx=0x7f93bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.384+0000 7f93c8c00640 1 -- 172.21.15.107:0/647439303 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc009700 con 0x7f93cc073520 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.384+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc19e720 con 0x7f93cc073520 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.384+0000 7f93c8c00640 1 -- 172.21.15.107:0/647439303 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93bc00cd80 con 0x7f93cc073520 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.384+0000 7f93c8c00640 1 -- 172.21.15.107:0/647439303 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0188e0 con 0x7f93cc073520 2024-03-11T19:53:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.384+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc1aa960 con 0x7f93cc073520 2024-03-11T19:53:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.387+0000 7f93c8c00640 1 -- 172.21.15.107:0/647439303 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f93bc010ba0 con 0x7f93cc073520 2024-03-11T19:53:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.387+0000 7f93d4f9f740 1 Processor -- start 2024-03-11T19:53:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.387+0000 7f93d4f9f740 1 -- start start 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbac3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbacf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efba7f620 con 0x559efbac3ff0 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efba318c0 con 0x559efbac4d60 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efba2edf0 con 0x559efbacd570 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34868/0 (socket says 172.21.15.107:34868) 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93cb400640 1 -- 172.21.15.107:0/2666339432 learned_addr learned my addr 172.21.15.107:0/2666339432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93cbe00640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbac3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.388+0000 7f93c3e00640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbacf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 -- 172.21.15.107:0/2666339432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 msgr2=0x559efbacf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbacf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 -- 172.21.15.107:0/2666339432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 msgr2=0x559efbac3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbac3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 -- 172.21.15.107:0/2666339432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efb9abb00 con 0x559efbac3ff0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cbe00640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbac3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93c8c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 0x7f93b00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cb400640 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0cb380 tx=0x7f93bc0d64b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a615eae93cf32f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93c8c00640 1 -- 172.21.15.107:0/647439303 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f93bc014070 con 0x7f93cc073520 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93cbe00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 0x7f93b00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93bbe00640 1 -- 172.21.15.107:0/2666339432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc096ae0 con 0x559efbac3ff0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93bbe00640 1 -- 172.21.15.107:0/2666339432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93bc09a610 con 0x559efbac3ff0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.389+0000 7f93bbe00640 1 -- 172.21.15.107:0/2666339432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc09a840 con 0x559efbac3ff0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 -- 172.21.15.107:0/2666339432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 msgr2=0x559efbac4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93cbe00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 0x7f93b00a40e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93cc06a6b0 tx=0x7f93c4009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0cb380 tx=0x7f93bc0d64b0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 -- 172.21.15.107:0/2666339432 shutdown_connections 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbacf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbac4470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/2666339432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbac3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 -- 172.21.15.107:0/2666339432 >> 172.21.15.107:0/2666339432 conn(0x559efba8b4b0 msgr2=0x559efbabece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 -- 172.21.15.107:0/2666339432 shutdown_connections 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.390+0000 7f93d4f9f740 1 -- 172.21.15.107:0/2666339432 wait complete. 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 -- start start 2024-03-11T19:53:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 Processor -- start 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbacd570 0x559efbacb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efba2a5d0 con 0x559efbacd570 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb9cfb50 con 0x559efbac4d60 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb9cf7f0 con 0x559efbac3ff0 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.391+0000 7f93d4f9f740 10 client.0 _ll_register_callbacks cb 0x559efb9e3550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbacd570 0x559efbacb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:43374/0 (socket says 172.21.15.107:43374) 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38900/0 (socket says 172.21.15.107:38900) 2024-03-11T19:53:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 -- 172.21.15.107:0/89396893 learned_addr learned my addr 172.21.15.107:0/89396893 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 -- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 msgr2=0x559efbacab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 -- 172.21.15.107:0/89396893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbacd570 msgr2=0x559efbacb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbacd570 0x559efbacb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 -- 172.21.15.107:0/89396893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc1aa960 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93c3e00640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93cb400640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93bc09a7d0 tx=0x7f93bc00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761a3f8012169c63 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.392+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc09aca0 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f93bc09e990 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559efb9fccd0 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93bc09a610 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 10 client.25537 fetch_fsmap learned FSMap version 24 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559efba28c50 con 0x559efbac3ff0 2024-03-11T19:53:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559efb9fccd0 con 0x559efbac3ff0 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93bc0a2b50 con 0x559efbac3ff0 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 10 client.25537 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 20 client.25537 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbadd240 0x559efbadf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 4 client.25537 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.393+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559efba20ce0 con 0x559efbadd240 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.394+0000 7f93c3e00640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbadd240 0x559efbadf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.394+0000 7f93c3e00640 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbadd240 0x559efbadf630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4194ec7a2de6c0 server_cookie=a57df67f4b91853 in_seq=0 out_seq=0 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.394+0000 7f93b8c00640 10 client.25537 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.395+0000 7f93b8c00640 1 -- 172.21.15.107:0/89396893 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559efba20ce0 con 0x559efbadd240 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.395+0000 7f93b8c00640 10 client.25537 handle_command_reply: tid=0 2024-03-11T19:53:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.395+0000 7f93d4f9f740 1 client.25537 shutdown 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 msgr2=0x559efbaca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93bc09a7d0 tx=0x7f93bc00cef0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 shutdown_connections 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbadd240 0x559efbadf630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbacab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbacd570 0x559efbacb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/89396893 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac3ff0 0x559efbaca5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93b8c00640 0 client.25537 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 >> 172.21.15.107:0/89396893 conn(0x559efba8b4b0 msgr2=0x559efbac0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 shutdown_connections 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 1 -- 172.21.15.107:0/89396893 wait complete. 2024-03-11T19:53:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.396+0000 7f93d4f9f740 20 client.25537 trim_cache size 0 max 16384 2024-03-11T19:53:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 Processor -- start 2024-03-11T19:53:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 -- start start 2024-03-11T19:53:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 0x559efbacab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbacd570 0x559efbacb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb772e60 con 0x559efbac3ff0 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efbacb590 con 0x559efbacd570 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.410+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efbabfd30 con 0x559efbac4d60 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 0x559efbacab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34882/0 (socket says 172.21.15.107:34882) 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93cb400640 1 -- 172.21.15.107:0/1698374395 learned_addr learned my addr 172.21.15.107:0/1698374395 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93cbe00640 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbacd570 0x559efbacb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 -- 172.21.15.107:0/1698374395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbacd570 msgr2=0x559efbacb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbacd570 0x559efbacb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 -- 172.21.15.107:0/1698374395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 msgr2=0x559efbacccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 -- 172.21.15.107:0/1698374395 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efb9c5530 con 0x559efbac4d60 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93cb400640 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 0x559efbacab10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fa4be14d2eaef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.411+0000 7f93bb400640 1 -- 172.21.15.107:0/1698374395 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4045040 con 0x559efbac4d60 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93bb400640 1 -- 172.21.15.107:0/1698374395 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93b4047040 con 0x559efbac4d60 2024-03-11T19:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93bb400640 1 -- 172.21.15.107:0/1698374395 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b402cd40 con 0x559efbac4d60 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1698374395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 msgr2=0x559efbacab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 0x559efbacab10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4009260 comp rx=0 tx=0).stop 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1698374395 shutdown_connections 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbacd570 0x559efbacb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbac4d60 0x559efbacab10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1698374395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbacccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1698374395 >> 172.21.15.107:0/1698374395 conn(0x559efba8b4b0 msgr2=0x559efbac8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1698374395 shutdown_connections 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.412+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1698374395 wait complete. 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 -- start start 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 Processor -- start 2024-03-11T19:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbad2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbad31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efbad3870 con 0x559efbac3ff0 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efbad39b0 con 0x559efbac4d60 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efbb336c0 con 0x559efbacd570 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93d4f9f740 10 client.0 _ll_register_callbacks cb 0x7f93c40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34896/0 (socket says 172.21.15.107:34896) 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.413+0000 7f93cb400640 1 -- 172.21.15.107:0/1353153257 learned_addr learned my addr 172.21.15.107:0/1353153257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbad2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cbe00640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbad31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 -- 172.21.15.107:0/1353153257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 msgr2=0x559efbad31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbad31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 -- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 msgr2=0x559efbad2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbad2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 -- 172.21.15.107:0/1353153257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efba28c50 con 0x559efbac3ff0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbad2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93cb400640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0a31a0 tx=0x7f93bc0a3030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa118c0c23d5ec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc09af40 con 0x559efbac3ff0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f93bc09e990 con 0x559efbac3ff0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.414+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559efba4fd30 con 0x559efbac3ff0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93bc0cb350 con 0x559efbac3ff0 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-03-11T19:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f93cc114c90 con 0x559efbac3ff0 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559efba4fd30 con 0x559efbac3ff0 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93bc09aaa0 con 0x559efbac3ff0 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 20 client.15420 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbb34a30 0x559efbb36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 4 client.15420 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.415+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559efb72a600 con 0x559efbb34a30 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.416+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbb34a30 0x559efbb36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.416+0000 7f93c3e00640 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbb34a30 0x559efbb36e50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32118ef37578f42 server_cookie=fa1ad4b4869dac5b in_seq=0 out_seq=0 2024-03-11T19:53:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.416+0000 7f93b8c00640 10 client.15420 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93b8c00640 1 -- 172.21.15.107:0/1353153257 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559efb72a600 con 0x559efbb34a30 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93b8c00640 10 client.15420 handle_command_reply: tid=0 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 client.15420 shutdown 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 msgr2=0x559efbad4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0a31a0 tx=0x7f93bc0a3030 comp rx=0 tx=0).stop 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 shutdown_connections 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x559efbb34a30 0x559efbb36e50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559efbac4d60 0x559efbad2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x559efbacd570 0x559efbad31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 --2- 172.21.15.107:0/1353153257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559efbac3ff0 0x559efbad4800 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93b8c00640 0 client.15420 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 >> 172.21.15.107:0/1353153257 conn(0x559efba8b4b0 msgr2=0x559efbabd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 shutdown_connections 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.430+0000 7f93d4f9f740 1 -- 172.21.15.107:0/1353153257 wait complete. 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d4f9f740 20 client.15420 trim_cache size 0 max 16384 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:53:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:53:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:53:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 msgr2=0x7f93b00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 0x7f93b00a40e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93cc06a6b0 tx=0x7f93c4009500 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 msgr2=0x7f93cc1a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00e9f0 tx=0x7f93bc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 shutdown_connections 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f93b00a1c20 0x7f93b00a40e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93cc073e40 0x7f93cc1a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc111f70 0x7f93cc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 --2- 172.21.15.107:0/647439303 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f93cc073520 0x7f93cc1a38c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.431+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 >> 172.21.15.107:0/647439303 conn(0x7f93cc0fca90 msgr2=0x7f93cc0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.432+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 shutdown_connections 2024-03-11T19:53:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:41.432+0000 7f93d0c00640 1 -- 172.21.15.107:0/647439303 wait complete. 2024-03-11T19:53:43.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:42 smithi122 ceph-mon[36397]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:43.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:42 smithi145 ceph-mon[36730]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:43.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:42 smithi107 ceph-mon[31712]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-11T19:53:45.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:44 smithi122 ceph-mon[36397]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-03-11T19:53:45.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:44 smithi145 ceph-mon[36730]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-03-11T19:53:45.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:44 smithi107 ceph-mon[31712]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-03-11T19:53:47.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:46 smithi122 ceph-mon[36397]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:47.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:46 smithi145 ceph-mon[36730]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:47.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:46 smithi107 ceph-mon[31712]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:49.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:48 smithi122 ceph-mon[36397]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:49.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:48 smithi145 ceph-mon[36730]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:49.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:48 smithi107 ceph-mon[31712]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:51.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:50 smithi122 ceph-mon[36397]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:51.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:50 smithi145 ceph-mon[36730]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:51.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:50 smithi107 ceph-mon[31712]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:53.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:52 smithi122 ceph-mon[36397]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:53.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:52 smithi145 ceph-mon[36730]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:53.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:52 smithi107 ceph-mon[31712]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.568+0000 7f077ea00640 1 -- 172.21.15.107:0/3415302731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073e40 msgr2=0x7f0778111a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.568+0000 7f077ea00640 1 --2- 172.21.15.107:0/3415302731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073e40 0x7f0778111a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f076402bcc0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 -- 172.21.15.107:0/3415302731 shutdown_connections 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 --2- 172.21.15.107:0/3415302731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f0778114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 --2- 172.21.15.107:0/3415302731 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073e40 0x7f0778111a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 --2- 172.21.15.107:0/3415302731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073520 0x7f0778073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 -- 172.21.15.107:0/3415302731 >> 172.21.15.107:0/3415302731 conn(0x7f07780fcad0 msgr2=0x7f07780feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 -- 172.21.15.107:0/3415302731 shutdown_connections 2024-03-11T19:53:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.569+0000 7f077ea00640 1 -- 172.21.15.107:0/3415302731 wait complete. 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 Processor -- start 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 -- start start 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 0x7f07781a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 0x7f07781a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f077819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819e110 con 0x7f0778111f70 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819e280 con 0x7f0778073e40 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077ea00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819e440 con 0x7f0778073520 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.570+0000 7f077e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f077819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 0x7f07781a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 0x7f07781a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f077819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42858/0 (socket says 172.21.15.107:42858) 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 0x7f07781a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:53588/0 (socket says 172.21.15.107:53588) 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077e000640 1 -- 172.21.15.107:0/2542654852 learned_addr learned my addr 172.21.15.107:0/2542654852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 -- 172.21.15.107:0/2542654852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 msgr2=0x7f07781a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 0x7f07781a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 -- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 msgr2=0x7f077819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f077819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 -- 172.21.15.107:0/2542654852 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009560 con 0x7f0778073e40 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.571+0000 7f077cc00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 0x7f07781a3e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0764036040 tx=0x7f076402c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.572+0000 7f0776000640 1 -- 172.21.15.107:0/2542654852 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764004350 con 0x7f0778073e40 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.572+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077819e6c0 con 0x7f0778073e40 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.572+0000 7f0776000640 1 -- 172.21.15.107:0/2542654852 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07640044b0 con 0x7f0778073e40 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.572+0000 7f0776000640 1 -- 172.21.15.107:0/2542654852 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764003d30 con 0x7f0778073e40 2024-03-11T19:53:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.572+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07781aa960 con 0x7f0778073e40 2024-03-11T19:53:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.575+0000 7f0776000640 1 -- 172.21.15.107:0/2542654852 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) v1 ==== 136456+0+0 (secure 0 0 0) 0x7f0764004df0 con 0x7f0778073e40 2024-03-11T19:53:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.575+0000 7f0782c63740 1 Processor -- start 2024-03-11T19:53:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 -- start start 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966732ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x55596673e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559666ee620 con 0x555966732ff0 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559666a08c0 con 0x555966733d60 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f0782c63740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596669ddf0 con 0x55596673c570 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42866/0 (socket says 172.21.15.107:42866) 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f077d600640 1 -- 172.21.15.107:0/3435818237 learned_addr learned my addr 172.21.15.107:0/3435818237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f077e000640 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966732ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.576+0000 7f077cc00640 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x55596673e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f0776000640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 0x7f07600a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 -- 172.21.15.107:0/3435818237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 msgr2=0x55596673e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x55596673e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 -- 172.21.15.107:0/3435818237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 msgr2=0x555966732ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966732ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 -- 172.21.15.107:0/3435818237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55596661ab00 con 0x555966732ff0 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f0776000640 1 -- 172.21.15.107:0/2542654852 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f07640f0240 con 0x7f0778073e40 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077e000640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 0x7f07600a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.577+0000 7f077d600640 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f076c009c90 tx=0x7f076c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c0d3f11886ce server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f076a600640 1 -- 172.21.15.107:0/3435818237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c005220 con 0x555966732ff0 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f077e000640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 0x7f07600a40e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0770009a40 tx=0x7f0770004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f076a600640 1 -- 172.21.15.107:0/3435818237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f076c004c80 con 0x555966732ff0 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f076a600640 1 -- 172.21.15.107:0/3435818237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c019810 con 0x555966732ff0 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f0782c63740 1 -- 172.21.15.107:0/3435818237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 msgr2=0x555966733470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.578+0000 7f0782c63740 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f076c009c90 tx=0x7f076c011810 comp rx=0 tx=0).stop 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 -- 172.21.15.107:0/3435818237 shutdown_connections 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x55596673e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966733470 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 --2- 172.21.15.107:0/3435818237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966732ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 -- 172.21.15.107:0/3435818237 >> 172.21.15.107:0/3435818237 conn(0x5559666fa4b0 msgr2=0x55596672dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 -- 172.21.15.107:0/3435818237 shutdown_connections 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 -- 172.21.15.107:0/3435818237 wait complete. 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 -- start start 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.579+0000 7f0782c63740 1 Processor -- start 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966739b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 0x55596673a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559666995d0 con 0x55596673c570 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596663eb50 con 0x555966732ff0 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55596663e7f0 con 0x555966733d60 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f0782c63740 10 client.0 _ll_register_callbacks cb 0x555966652550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 0x55596673a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38172/0 (socket says 172.21.15.107:38172) 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077d600640 1 -- 172.21.15.107:0/3613611061 learned_addr learned my addr 172.21.15.107:0/3613611061 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077cc00640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966739b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077d600640 1 -- 172.21.15.107:0/3613611061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 msgr2=0x555966739b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.580+0000 7f077d600640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966739b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077d600640 1 -- 172.21.15.107:0/3613611061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 msgr2=0x55596673a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077d600640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 0x55596673a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077d600640 1 -- 172.21.15.107:0/3613611061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c0098f0 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077cc00640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966739b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077e000640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 0x55596673a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f077d600640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f076c019b00 tx=0x7f076c019b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85f53d533779897 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f075ea00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c023630 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f076c005430 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.581+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55596666bcd0 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f076c023790 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 10 client.25547 fetch_fsmap learned FSMap version 24 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555966697c50 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55596666bcd0 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f076c021040 con 0x555966732ff0 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 10 client.25547 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:53:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 20 client.25547 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55596674c490 0x55596674e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 4 client.25547 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.582+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55596668fce0 con 0x55596674c490 2024-03-11T19:53:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.583+0000 7f077cc00640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55596674c490 0x55596674e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.583+0000 7f075ea00640 10 client.25547 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.583+0000 7f077cc00640 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55596674c490 0x55596674e880 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1a403eec793df4 server_cookie=959c4d4a3abddc0b in_seq=0 out_seq=0 2024-03-11T19:53:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f075ea00640 1 -- 172.21.15.107:0/3613611061 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55596668fce0 con 0x55596674c490 2024-03-11T19:53:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f075ea00640 10 client.25547 handle_command_reply: tid=0 2024-03-11T19:53:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 client.25547 shutdown 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 msgr2=0x5559667395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f076c019b00 tx=0x7f076c019b30 comp rx=0 tx=0).stop 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 shutdown_connections 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55596674c490 0x55596674e880 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55596673c570 0x55596673a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966739b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 --2- 172.21.15.107:0/3613611061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966732ff0 0x5559667395d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f075ea00640 0 client.25547 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 >> 172.21.15.107:0/3613611061 conn(0x5559666fa4b0 msgr2=0x55596672fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 shutdown_connections 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 1 -- 172.21.15.107:0/3613611061 wait complete. 2024-03-11T19:53:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.584+0000 7f0782c63740 20 client.25547 trim_cache size 0 max 16384 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.598+0000 7f0782c63740 1 Processor -- start 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.598+0000 7f0782c63740 1 -- start start 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966737ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55596673c570 0x5559667383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559663e1e60 con 0x555966732ff0 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559667295a0 con 0x55596673c570 2024-03-11T19:53:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f0782c63740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555966738920 con 0x555966733d60 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42874/0 (socket says 172.21.15.107:42874) 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077d600640 1 -- 172.21.15.107:0/2173295510 learned_addr learned my addr 172.21.15.107:0/2173295510 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077e000640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55596673c570 0x5559667383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966737ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 -- 172.21.15.107:0/2173295510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55596673c570 msgr2=0x5559667383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55596673c570 0x5559667383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 -- 172.21.15.107:0/2173295510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 msgr2=0x55596673bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077cc00640 1 -- 172.21.15.107:0/2173295510 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555966634530 con 0x555966733d60 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.599+0000 7f077d600640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.600+0000 7f077cc00640 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966737ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07640ed620 tx=0x7f07640f2710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a0182a84a71cb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.600+0000 7f0769c00640 1 -- 172.21.15.107:0/2173295510 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640bc3b0 con 0x555966733d60 2024-03-11T19:53:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.600+0000 7f0769c00640 1 -- 172.21.15.107:0/2173295510 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07640bd150 con 0x555966733d60 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.600+0000 7f0769c00640 1 -- 172.21.15.107:0/2173295510 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640055d0 con 0x555966733d60 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- 172.21.15.107:0/2173295510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 msgr2=0x555966737ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966737ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07640ed620 tx=0x7f07640f2710 comp rx=0 tx=0).stop 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- 172.21.15.107:0/2173295510 shutdown_connections 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55596673c570 0x5559667383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555966733d60 0x555966737ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 --2- 172.21.15.107:0/2173295510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x55596673bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- 172.21.15.107:0/2173295510 >> 172.21.15.107:0/2173295510 conn(0x5559666fa4b0 msgr2=0x55596672d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- 172.21.15.107:0/2173295510 shutdown_connections 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- 172.21.15.107:0/2173295510 wait complete. 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 -- start start 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.601+0000 7f0782c63740 1 Processor -- start 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966741920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x555966741e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559667423a0 con 0x555966732ff0 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x555966742510 con 0x555966733d60 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555966742680 con 0x55596673c570 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f0782c63740 10 client.0 _ll_register_callbacks cb 0x7f0770009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42876/0 (socket says 172.21.15.107:42876) 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 -- 172.21.15.107:0/2324719287 learned_addr learned my addr 172.21.15.107:0/2324719287 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077cc00640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966741920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077e000640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x555966741e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 -- 172.21.15.107:0/2324719287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 msgr2=0x555966741e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x555966741e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 -- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 msgr2=0x555966741920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966741920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.602+0000 7f077d600640 1 -- 172.21.15.107:0/2324719287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555966697c50 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f077d600640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f076c005690 tx=0x7f076c0225c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd62b353e04f3572 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f075ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c022ee0 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f076c000e40 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559666bed30 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f076c022ae0 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f0782c63740 10 client.15438 fetch_fsmap learned FSMap version 24 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.603+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f07781aa960 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f076c03e080 con 0x555966732ff0 2024-03-11T19:53:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f076c0039a0 con 0x555966732ff0 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 10 client.15438 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 20 client.15438 populate_metadata read hostname 'smithi107' 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5559667a3a90 0x5559667a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 4 client.15438 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555966399600 con 0x5559667a3a90 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.604+0000 7f077cc00640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5559667a3a90 0x5559667a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.605+0000 7f075ea00640 10 client.15438 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.605+0000 7f077cc00640 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5559667a3a90 0x5559667a5e80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a0e1a2b5f7f6e1 server_cookie=493bd8954b222bc9 in_seq=0 out_seq=0 2024-03-11T19:53:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.611+0000 7f075ea00640 1 -- 172.21.15.107:0/2324719287 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555966399600 con 0x5559667a3a90 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.611+0000 7f075ea00640 10 client.15438 handle_command_reply: tid=0 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.611+0000 7f0782c63740 1 client.15438 shutdown 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 msgr2=0x555966743580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f076c005690 tx=0x7f076c0225c0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 shutdown_connections 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5559667a3a90 0x5559667a5e80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555966733d60 0x555966741920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55596673c570 0x555966741e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 --2- 172.21.15.107:0/2324719287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555966732ff0 0x555966743580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f075ea00640 0 client.15438 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 >> 172.21.15.107:0/2324719287 conn(0x5559666fa4b0 msgr2=0x55596672d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 shutdown_connections 2024-03-11T19:53:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 1 -- 172.21.15.107:0/2324719287 wait complete. 2024-03-11T19:53:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.612+0000 7f0782c63740 20 client.15438 trim_cache size 0 max 16384 2024-03-11T19:53:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:53:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:53:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:53:54.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 msgr2=0x7f07600a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 0x7f07600a40e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0770009a40 tx=0x7f0770004a40 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 msgr2=0x7f07781a3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 0x7f07781a3e00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0764036040 tx=0x7f076402c710 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 shutdown_connections 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f07600a1c20 0x7f07600a40e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0778073e40 0x7f07781a3e00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0778111f70 0x7f077819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 --2- 172.21.15.107:0/2542654852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0778073520 0x7f07781a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 >> 172.21.15.107:0/2542654852 conn(0x7f07780fcad0 msgr2=0x7f07780fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 shutdown_connections 2024-03-11T19:53:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:53:54.613+0000 7f077ea00640 1 -- 172.21.15.107:0/2542654852 wait complete. 2024-03-11T19:53:55.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:54 smithi122 ceph-mon[36397]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:55.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:54 smithi145 ceph-mon[36730]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:55.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:54 smithi107 ceph-mon[31712]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-11T19:53:57.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:56 smithi122 ceph-mon[36397]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:53:57.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:56 smithi145 ceph-mon[36730]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:53:57.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:56 smithi107 ceph-mon[31712]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T19:53:59.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:53:58 smithi122 ceph-mon[36397]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-11T19:53:59.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:53:58 smithi145 ceph-mon[36730]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-11T19:53:59.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:53:58 smithi107 ceph-mon[31712]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-11T19:54:00.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:54:00.094+0000 7f9b09e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:54:00.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:54:00.094+0000 7f9b09e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:54:00.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:54:00.094+0000 7f9b09e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:54:00.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:54:00.094+0000 7f9b09e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:54:00.248 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T19:54:00.094+0000 7f9b09e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:54:01.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:00 smithi122 ceph-mon[36397]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:01.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:00 smithi145 ceph-mon[36730]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:00 smithi107 ceph-mon[31712]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:02 smithi122 ceph-mon[36397]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:03.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:02 smithi122 ceph-mon[36397]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-03-11T19:54:03.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:02 smithi145 ceph-mon[36730]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:03.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:02 smithi145 ceph-mon[36730]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-03-11T19:54:03.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:02 smithi107 ceph-mon[31712]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:03.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:02 smithi107 ceph-mon[31712]: mgrmap e25: x(active, since 36s), standbys: y, z 2024-03-11T19:54:05.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:04 smithi122 ceph-mon[36397]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:54:05.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:04 smithi145 ceph-mon[36730]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:54:05.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:04 smithi107 ceph-mon[31712]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-11T19:54:07.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:06 smithi122 ceph-mon[36397]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:54:07.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:06 smithi145 ceph-mon[36730]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:54:07.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:06 smithi107 ceph-mon[31712]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-11T19:54:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 -- 172.21.15.107:0/1693804550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 msgr2=0x7f78741015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 --2- 172.21.15.107:0/1693804550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f78741015b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7864009930 tx=0x7f786402bc20 comp rx=0 tx=0).stop 2024-03-11T19:54:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 -- 172.21.15.107:0/1693804550 shutdown_connections 2024-03-11T19:54:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 --2- 172.21.15.107:0/1693804550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7874101b80 0x7f7874105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 --2- 172.21.15.107:0/1693804550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f78741062a0 0x7f787410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 --2- 172.21.15.107:0/1693804550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f78741015b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 -- 172.21.15.107:0/1693804550 >> 172.21.15.107:0/1693804550 conn(0x7f78740fcad0 msgr2=0x7f78740feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 -- 172.21.15.107:0/1693804550 shutdown_connections 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.764+0000 7f7873e00640 1 -- 172.21.15.107:0/1693804550 wait complete. 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 Processor -- start 2024-03-11T19:54:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 -- start start 2024-03-11T19:54:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 0x7f7874197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 0x7f787419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787419c9c0 con 0x7f78741011d0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787419cb30 con 0x7f78741062a0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.766+0000 7f7873e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787419cca0 con 0x7f787419bfd0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44484/0 (socket says 172.21.15.107:44484) 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 -- 172.21.15.107:0/487377060 learned_addr learned my addr 172.21.15.107:0/487377060 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7873400640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 0x7f787419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872000640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 0x7f7874197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 -- 172.21.15.107:0/487377060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 msgr2=0x7f787419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 0x7f787419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 -- 172.21.15.107:0/487377060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 msgr2=0x7f7874197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 0x7f7874197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 -- 172.21.15.107:0/487377060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860009490 con 0x7f78741011d0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7873400640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 0x7f787419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872000640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 0x7f7874197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.767+0000 7f7872a00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f786402c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.768+0000 7f786b400640 1 -- 172.21.15.107:0/487377060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786402dea0 con 0x7f78741011d0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.768+0000 7f786b400640 1 -- 172.21.15.107:0/487377060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78640042d0 con 0x7f78741011d0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.768+0000 7f786b400640 1 -- 172.21.15.107:0/487377060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864003a10 con 0x7f78741011d0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.768+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c009840 con 0x7f78741011d0 2024-03-11T19:54:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.768+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7864009560 con 0x7f78741011d0 2024-03-11T19:54:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.772+0000 7f787c5ef740 1 Processor -- start 2024-03-11T19:54:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.772+0000 7f787c5ef740 1 -- start start 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de6ff0 0x5632f0de7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0df6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0da2620 con 0x5632f0de7d60 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0d548c0 con 0x5632f0df4980 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f787c5ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0d51df0 con 0x5632f0de6ff0 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f786b400640 1 -- 172.21.15.107:0/487377060 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f7864003b70 con 0x7f78741011d0 2024-03-11T19:54:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f7873400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f7873400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44496/0 (socket says 172.21.15.107:44496) 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f7873400640 1 -- 172.21.15.107:0/273083639 learned_addr learned my addr 172.21.15.107:0/273083639 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f7872000640 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0df6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.773+0000 7f7872a00640 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de6ff0 0x5632f0de7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 -- 172.21.15.107:0/273083639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de6ff0 msgr2=0x5632f0de7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de6ff0 0x5632f0de7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 -- 172.21.15.107:0/273083639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 msgr2=0x5632f0df6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0df6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 -- 172.21.15.107:0/273083639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f0cceb00 con 0x5632f0de7d60 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f7873400640 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e890 tx=0x7f786c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f77f7fe9cb0cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f785fe00640 1 -- 172.21.15.107:0/273083639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c011ce0 con 0x5632f0de7d60 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f785fe00640 1 -- 172.21.15.107:0/273083639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f786c003e50 con 0x5632f0de7d60 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.774+0000 7f785fe00640 1 -- 172.21.15.107:0/273083639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c019430 con 0x5632f0de7d60 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 -- 172.21.15.107:0/273083639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 msgr2=0x5632f0de6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e890 tx=0x7f786c00ed50 comp rx=0 tx=0).stop 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f786b400640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 0x7f78580a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 -- 172.21.15.107:0/273083639 shutdown_connections 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0df6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de6ff0 0x5632f0de7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 --2- 172.21.15.107:0/273083639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de7d60 0x5632f0de6ab0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 -- 172.21.15.107:0/273083639 >> 172.21.15.107:0/273083639 conn(0x5632f0dae4b0 msgr2=0x5632f0de1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f787c5ef740 1 -- 172.21.15.107:0/273083639 shutdown_connections 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f786b400640 1 -- 172.21.15.107:0/487377060 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f78640f0270 con 0x7f78741011d0 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.775+0000 7f7873400640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 0x7f78580a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.776+0000 7f787c5ef740 1 -- 172.21.15.107:0/273083639 wait complete. 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.776+0000 7f787c5ef740 1 -- start start 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.776+0000 7f7873400640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 0x7f78580a41f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e860 tx=0x7f786c009b10 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.776+0000 7f787c5ef740 1 Processor -- start 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0df1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 0x5632f0df2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0d4d5d0 con 0x5632f0df4980 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0cf2b50 con 0x5632f0de6ff0 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0cf27f0 con 0x5632f0de7d60 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f787c5ef740 10 client.0 _ll_register_callbacks cb 0x5632f0d06550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7872a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7873400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 0x5632f0df2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7872a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:45814/0 (socket says 172.21.15.107:45814) 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7873400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 0x5632f0df2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44500/0 (socket says 172.21.15.107:44500) 2024-03-11T19:54:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7872a00640 1 -- 172.21.15.107:0/3171261959 learned_addr learned my addr 172.21.15.107:0/3171261959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.777+0000 7f7872000640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0df1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 -- 172.21.15.107:0/3171261959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 msgr2=0x5632f0df1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0df1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 -- 172.21.15.107:0/3171261959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 msgr2=0x5632f0df2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 0x5632f0df2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 -- 172.21.15.107:0/3171261959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c00e540 con 0x5632f0de6ff0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f7872a00640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7864006010 tx=0x7f78640bc480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17bcec3a89b98d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f785cc00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864003e20 con 0x5632f0de6ff0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f78640bd040 con 0x5632f0de6ff0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632f0d1fcd0 con 0x5632f0de6ff0 2024-03-11T19:54:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78640bc940 con 0x5632f0de6ff0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f787c5ef740 10 client.25568 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.778+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5632f0d4bc50 con 0x5632f0de6ff0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5632f0d1fcd0 con 0x5632f0de6ff0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f786403c070 con 0x5632f0de6ff0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 10 client.25568 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 20 client.25568 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e04770 0x5632f0e06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 4 client.25568 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632f0d43ce0 con 0x5632f0e04770 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.779+0000 7f7872000640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e04770 0x5632f0e06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.780+0000 7f7872000640 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e04770 0x5632f0e06b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36300f07c1962b32 server_cookie=9b6893006f762a99 in_seq=0 out_seq=0 2024-03-11T19:54:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.780+0000 7f785cc00640 10 client.25568 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f785cc00640 1 -- 172.21.15.107:0/3171261959 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632f0d43ce0 con 0x5632f0e04770 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f785cc00640 10 client.25568 handle_command_reply: tid=0 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 client.25568 shutdown 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 msgr2=0x5632f0df19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7864006010 tx=0x7f78640bc480 comp rx=0 tx=0).stop 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 shutdown_connections 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e04770 0x5632f0e06b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0df4980 0x5632f0df2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0df1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f785cc00640 0 client.25568 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 --2- 172.21.15.107:0/3171261959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de6ff0 0x5632f0df19b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 >> 172.21.15.107:0/3171261959 conn(0x5632f0dae4b0 msgr2=0x5632f0de3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 shutdown_connections 2024-03-11T19:54:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 1 -- 172.21.15.107:0/3171261959 wait complete. 2024-03-11T19:54:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.781+0000 7f787c5ef740 20 client.25568 trim_cache size 0 max 16384 2024-03-11T19:54:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.795+0000 7f787c5ef740 1 Processor -- start 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.795+0000 7f787c5ef740 1 -- start start 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 0x5632f0df2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0df4980 0x5632f0deec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0a95e60 con 0x5632f0de6ff0 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0e045a0 con 0x5632f0de7d60 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f787c5ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0de83d0 con 0x5632f0df4980 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44506/0 (socket says 172.21.15.107:44506) 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 -- 172.21.15.107:0/441011760 learned_addr learned my addr 172.21.15.107:0/441011760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7873400640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0df4980 0x5632f0deec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872000640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 0x5632f0df2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 -- 172.21.15.107:0/441011760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0df4980 msgr2=0x5632f0deec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0df4980 0x5632f0deec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 -- 172.21.15.107:0/441011760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 msgr2=0x5632f0df2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 0x5632f0df2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872a00640 1 -- 172.21.15.107:0/441011760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f0ce8530 con 0x5632f0de6ff0 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.796+0000 7f7872000640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 0x5632f0df2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.797+0000 7f7872a00640 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7864005c20 tx=0x7f78640b8d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a066bacd6f4b97fa server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.797+0000 7f785f400640 1 -- 172.21.15.107:0/441011760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640d5440 con 0x5632f0de6ff0 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.797+0000 7f785f400640 1 -- 172.21.15.107:0/441011760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78640bd040 con 0x5632f0de6ff0 2024-03-11T19:54:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.797+0000 7f785f400640 1 -- 172.21.15.107:0/441011760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640c50e0 con 0x5632f0de6ff0 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- 172.21.15.107:0/441011760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 msgr2=0x5632f0df1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7864005c20 tx=0x7f78640b8d10 comp rx=0 tx=0).stop 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- 172.21.15.107:0/441011760 shutdown_connections 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0de7d60 0x5632f0df2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0df4980 0x5632f0deec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 --2- 172.21.15.107:0/441011760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0df1f00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- 172.21.15.107:0/441011760 >> 172.21.15.107:0/441011760 conn(0x5632f0dae4b0 msgr2=0x5632f0df0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- 172.21.15.107:0/441011760 shutdown_connections 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- 172.21.15.107:0/441011760 wait complete. 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.798+0000 7f787c5ef740 1 -- start start 2024-03-11T19:54:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 Processor -- start 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0dfa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0dfae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0e5ad30 con 0x5632f0de6ff0 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0e5aea0 con 0x5632f0df4980 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f0e5b060 con 0x5632f0de7d60 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f787c5ef740 10 client.0 _ll_register_callbacks cb 0x7f78640b8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f7872000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f7872000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:54216/0 (socket says 172.21.15.107:54216) 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f7872000640 1 -- 172.21.15.107:0/2254793481 learned_addr learned my addr 172.21.15.107:0/2254793481 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f7872a00640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0dfa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.799+0000 7f7873400640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0dfae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 -- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 msgr2=0x5632f0dfae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0dfae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 -- 172.21.15.107:0/2254793481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 msgr2=0x5632f0dfa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0dfa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 -- 172.21.15.107:0/2254793481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632f0d4bc50 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f7872000640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5632f0de7a90 tx=0x7f7860013d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=246cd33037874489 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f785cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860017030 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632f0d72d30 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.800+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7860010c70 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7860010e30 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 10 client.25585 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5632f0d0e2c0 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5632f0d72d30 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f786001e040 con 0x5632f0de7d60 2024-03-11T19:54:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 10 client.25585 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 20 client.25585 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e5c8c0 0x5632f0e5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 4 client.25585 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.801+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632f0a4d600 con 0x5632f0e5c8c0 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.802+0000 7f7872a00640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e5c8c0 0x5632f0e5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.802+0000 7f7872a00640 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e5c8c0 0x5632f0e5ece0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8257be2e93b1162f server_cookie=32c26ef72afd2fc3 in_seq=0 out_seq=0 2024-03-11T19:54:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.802+0000 7f785cc00640 10 client.25585 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f785cc00640 1 -- 172.21.15.107:0/2254793481 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632f0a4d600 con 0x5632f0e5c8c0 2024-03-11T19:54:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f785cc00640 10 client.25585 handle_command_reply: tid=0 2024-03-11T19:54:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 client.25585 shutdown 2024-03-11T19:54:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 msgr2=0x5632f0dfa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5632f0de7a90 tx=0x7f7860013d50 comp rx=0 tx=0).stop 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 shutdown_connections 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5632f0e5c8c0 0x5632f0e5ece0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632f0df4980 0x5632f0dfae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5632f0de7d60 0x5632f0dfa940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f785cc00640 0 client.25585 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 --2- 172.21.15.107:0/2254793481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632f0de6ff0 0x5632f0dfa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 >> 172.21.15.107:0/2254793481 conn(0x5632f0dae4b0 msgr2=0x5632f0de30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 shutdown_connections 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 1 -- 172.21.15.107:0/2254793481 wait complete. 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.810+0000 7f787c5ef740 20 client.25585 trim_cache size 0 max 16384 2024-03-11T19:54:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:54:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:54:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:54:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:54:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 msgr2=0x7f78580a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 0x7f78580a41f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e860 tx=0x7f786c009b10 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 msgr2=0x7f7874197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f786402c5e0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 shutdown_connections 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f78580a1d30 0x7f78580a41f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78741062a0 0x7f7874197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f787419bfd0 0x7f787419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 --2- 172.21.15.107:0/487377060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78741011d0 0x7f7874197710 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.811+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 >> 172.21.15.107:0/487377060 conn(0x7f78740fcad0 msgr2=0x7f78740fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.812+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 shutdown_connections 2024-03-11T19:54:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:07.812+0000 7f7873e00640 1 -- 172.21.15.107:0/487377060 wait complete. 2024-03-11T19:54:09.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:08 smithi122 ceph-mon[36397]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-11T19:54:09.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:08 smithi145 ceph-mon[36730]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-11T19:54:09.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:08 smithi107 ceph-mon[31712]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-11T19:54:10.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:10 smithi107 ceph-mon[31712]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:11.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:10 smithi122 ceph-mon[36397]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:11.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:10 smithi145 ceph-mon[36730]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:13.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:12 smithi122 ceph-mon[36397]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:13.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:12 smithi145 ceph-mon[36730]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:13.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:12 smithi107 ceph-mon[31712]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-03-11T19:54:15.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:14 smithi122 ceph-mon[36397]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-11T19:54:15.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:14 smithi145 ceph-mon[36730]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-11T19:54:15.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:14 smithi107 ceph-mon[31712]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-03-11T19:54:17.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:16 smithi122 ceph-mon[36397]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-11T19:54:17.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:16 smithi145 ceph-mon[36730]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-11T19:54:17.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:16 smithi107 ceph-mon[31712]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-11T19:54:19.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:18 smithi122 ceph-mon[36397]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-03-11T19:54:19.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:18 smithi145 ceph-mon[36730]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-03-11T19:54:19.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:18 smithi107 ceph-mon[31712]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.992+0000 7f0ed4a00640 1 -- 172.21.15.107:0/1492118910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed0073ed0 msgr2=0x7f0ed010d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.992+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/1492118910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed0073ed0 0x7f0ed010d260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009860 tx=0x7f0ebc02bc20 comp rx=0 tx=0).stop 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 -- 172.21.15.107:0/1492118910 shutdown_connections 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/1492118910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed0073ed0 0x7f0ed010d260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/1492118910 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ed010d7a0 0x7f0ed010fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/1492118910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed00735b0 0x7f0ed0073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 -- 172.21.15.107:0/1492118910 >> 172.21.15.107:0/1492118910 conn(0x7f0ed00fcad0 msgr2=0x7f0ed00feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 -- 172.21.15.107:0/1492118910 shutdown_connections 2024-03-11T19:54:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.993+0000 7f0ed4a00640 1 -- 172.21.15.107:0/1492118910 wait complete. 2024-03-11T19:54:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.994+0000 7f0ed4a00640 1 Processor -- start 2024-03-11T19:54:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.994+0000 7f0ed4a00640 1 -- start start 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ed0073ed0 0x7f0ed019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 0x7f0ed0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0199ea0 con 0x7f0ed010d7a0 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed019a010 con 0x7f0ed00735b0 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ed4a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed019a1d0 con 0x7f0ed0073ed0 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.995+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:52044/0 (socket says 172.21.15.107:52044) 2024-03-11T19:54:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 -- 172.21.15.107:0/355145009 learned_addr learned my addr 172.21.15.107:0/355145009 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 0x7f0ed0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 -- 172.21.15.107:0/355145009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ed0073ed0 msgr2=0x7f0ed019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ed0073ed0 0x7f0ed019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 -- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 msgr2=0x7f0ed0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 0x7f0ed0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecf400640 1 -- 172.21.15.107:0/355145009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0009560 con 0x7f0ed00735b0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.996+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 0x7f0ed0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ecf400640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000e9c0 tx=0x7f0ec000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ec7e00640 1 -- 172.21.15.107:0/355145009 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000cde0 con 0x7f0ed00735b0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ec7e00640 1 -- 172.21.15.107:0/355145009 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec00035f0 con 0x7f0ed00735b0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc009490 con 0x7f0ed00735b0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ec7e00640 1 -- 172.21.15.107:0/355145009 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0010670 con 0x7f0ed00735b0 2024-03-11T19:54:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:20.997+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed01a6580 con 0x7f0ed00735b0 2024-03-11T19:54:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.000+0000 7f0ec7e00640 1 -- 172.21.15.107:0/355145009 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f0ec0003ad0 con 0x7f0ed00735b0 2024-03-11T19:54:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.002+0000 7f0ed8da4740 1 Processor -- start 2024-03-11T19:54:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.002+0000 7f0ec7e00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 0x7f0eb40a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.002+0000 7f0ec7e00640 1 -- 172.21.15.107:0/355145009 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0ec0014070 con 0x7f0ed00735b0 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.002+0000 7f0ed8da4740 1 -- start start 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.002+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 0x7f0eb40a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797563470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x56079756e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079751e620 con 0x560797563d60 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607974d08c0 con 0x56079756c570 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607974cddf0 con 0x560797562ff0 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38860/0 (socket says 172.21.15.107:38860) 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecf400640 1 -- 172.21.15.107:0/3649804144 learned_addr learned my addr 172.21.15.107:0/3649804144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecea00640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797563470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x56079756e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.003+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 0x7f0eb40a41f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005fd0 tx=0x7f0ec8009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 -- 172.21.15.107:0/3649804144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 msgr2=0x560797563470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797563470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 -- 172.21.15.107:0/3649804144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 msgr2=0x56079756e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x56079756e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 -- 172.21.15.107:0/3649804144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56079744ab00 con 0x560797563d60 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x56079756e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ecf400640 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c6a0 tx=0x7f0ec009a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab850806a7b607b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ebbe00640 1 -- 172.21.15.107:0/3649804144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00ce160 con 0x560797563d60 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ebbe00640 1 -- 172.21.15.107:0/3649804144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec009a900 con 0x560797563d60 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.004+0000 7f0ebbe00640 1 -- 172.21.15.107:0/3649804144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00cb7d0 con 0x560797563d60 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 -- 172.21.15.107:0/3649804144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 msgr2=0x560797562ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c6a0 tx=0x7f0ec009a260 comp rx=0 tx=0).stop 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 -- 172.21.15.107:0/3649804144 shutdown_connections 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x56079756e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797563470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/3649804144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x560797562ab0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 -- 172.21.15.107:0/3649804144 >> 172.21.15.107:0/3649804144 conn(0x56079752a4b0 msgr2=0x56079755dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.005+0000 7f0ed8da4740 1 -- 172.21.15.107:0/3649804144 shutdown_connections 2024-03-11T19:54:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.006+0000 7f0ed8da4740 1 -- 172.21.15.107:0/3649804144 wait complete. 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.006+0000 7f0ed8da4740 1 -- start start 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.006+0000 7f0ed8da4740 1 Processor -- start 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x560797569630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079756a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607974c95d0 con 0x560797562ff0 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079746eb50 con 0x560797563d60 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079746e7f0 con 0x56079756c570 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ed8da4740 10 client.0 _ll_register_callbacks cb 0x560797482550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:52062/0 (socket says 172.21.15.107:52062) 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 -- 172.21.15.107:0/2380751018 learned_addr learned my addr 172.21.15.107:0/2380751018 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x560797569630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079756a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 -- 172.21.15.107:0/2380751018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 msgr2=0x56079756a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079756a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 -- 172.21.15.107:0/2380751018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 msgr2=0x560797569630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x560797569630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecea00640 1 -- 172.21.15.107:0/2380751018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed01a6580 con 0x560797563d60 2024-03-11T19:54:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.007+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079756a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.008+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ed019af10 tx=0x7f0ebc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740bd46a191f63eb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.008+0000 7f0eb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.008+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc005040 con 0x560797563d60 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.008+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56079749bcd0 con 0x560797563d60 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.008+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0ebc0051a0 con 0x560797563d60 2024-03-11T19:54:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.009+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ebc005360 con 0x560797563d60 2024-03-11T19:54:21.015 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:20 smithi122 ceph-mon[36397]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 10 client.25586 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5607974c7c50 con 0x560797563d60 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56079749bcd0 con 0x560797563d60 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ebc0046c0 con 0x560797563d60 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 10 client.25586 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 20 client.25586 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x560797584e20 0x560797587240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 4 client.25586 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.010+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607974bfce0 con 0x560797584e20 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.011+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x560797584e20 0x560797587240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.011+0000 7f0eb8c00640 10 client.25586 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.011+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x560797584e20 0x560797587240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3947d1264ec9e6af server_cookie=a30159f34f6e0da2 in_seq=0 out_seq=0 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2380751018 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607974bfce0 con 0x560797584e20 2024-03-11T19:54:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0eb8c00640 10 client.25586 handle_command_reply: tid=0 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 client.25586 shutdown 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 msgr2=0x560797569b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ed019af10 tx=0x7f0ebc004270 comp rx=0 tx=0).stop 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 shutdown_connections 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x560797584e20 0x560797587240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x560797569b70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079756a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2380751018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x560797569630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.012+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 >> 172.21.15.107:0/2380751018 conn(0x56079752a4b0 msgr2=0x56079755fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.013+0000 7f0eb8c00640 0 client.25586 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.013+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 shutdown_connections 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.013+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2380751018 wait complete. 2024-03-11T19:54:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.013+0000 7f0ed8da4740 20 client.25586 trim_cache size 0 max 16384 2024-03-11T19:54:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 Processor -- start 2024-03-11T19:54:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 -- start start 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x56079756bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x56079755d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079755dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797211e60 con 0x560797562ff0 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079755e290 con 0x560797563d60 2024-03-11T19:54:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.027+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079755eae0 con 0x56079756c570 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x56079756bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079755dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x56079755d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079755dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48118/0 (socket says 172.21.15.107:48118) 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x56079756bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38866/0 (socket says 172.21.15.107:38866) 2024-03-11T19:54:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecfe00640 1 -- 172.21.15.107:0/2146672044 learned_addr learned my addr 172.21.15.107:0/2146672044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 -- 172.21.15.107:0/2146672044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 msgr2=0x56079755dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079755dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 -- 172.21.15.107:0/2146672044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 msgr2=0x56079756bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x56079756bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 -- 172.21.15.107:0/2146672044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560797464530 con 0x560797563d60 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.028+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x56079755d810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc02c160 tx=0x7f0ebc0042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae16271e286e686e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ebb400640 1 -- 172.21.15.107:0/2146672044 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc02c320 con 0x560797563d60 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ebb400640 1 -- 172.21.15.107:0/2146672044 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc004c40 con 0x560797563d60 2024-03-11T19:54:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ebb400640 1 -- 172.21.15.107:0/2146672044 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc034920 con 0x560797563d60 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2146672044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 msgr2=0x56079755d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x56079755d810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc02c160 tx=0x7f0ebc0042e0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2146672044 shutdown_connections 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560797563d60 0x56079755d810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56079756c570 0x56079755dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2146672044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797562ff0 0x56079756bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.029+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2146672044 >> 172.21.15.107:0/2146672044 conn(0x56079752a4b0 msgr2=0x560797564930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.030+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2146672044 shutdown_connections 2024-03-11T19:54:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.030+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2146672044 wait complete. 2024-03-11T19:54:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.030+0000 7f0ed8da4740 1 -- start start 2024-03-11T19:54:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.030+0000 7f0ed8da4740 1 Processor -- start 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797579e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x5607975da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607975da880 con 0x560797563d60 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607975da9c0 con 0x56079756c570 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607975dab00 con 0x560797562ff0 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ed8da4740 10 client.0 _ll_register_callbacks cb 0x7f0ec0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797579e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38878/0 (socket says 172.21.15.107:38878) 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecea00640 1 -- 172.21.15.107:0/2261457580 learned_addr learned my addr 172.21.15.107:0/2261457580 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecfe00640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x5607975da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecf400640 1 -- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 msgr2=0x5607975da340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.031+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x5607975da340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ecf400640 1 -- 172.21.15.107:0/2261457580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 msgr2=0x56079757a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ecf400640 1 -- 172.21.15.107:0/2261457580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607974c7c50 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ecf400640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797579e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0096d60 tx=0x7f0ec009a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19991d5cc071b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0eb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec009ea50 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607974eed30 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0ec00a30a0 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec0096f70 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ed8da4740 10 client.25600 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.032+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56079748a2c0 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5607974eed30 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0ec0080020 con 0x560797562ff0 2024-03-11T19:54:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 10 client.25600 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 20 client.25600 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5607975dbe40 0x5607975de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 4 client.25600 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:54:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607971c9600 con 0x5607975dbe40 2024-03-11T19:54:21.040 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:20 smithi145 ceph-mon[36730]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.033+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5607975dbe40 0x5607975de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.034+0000 7f0ecea00640 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5607975dbe40 0x5607975de230 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78bb1b1ea522c81 server_cookie=e91e486861d228f7 in_seq=0 out_seq=0 2024-03-11T19:54:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.034+0000 7f0eb8c00640 10 client.25600 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.037+0000 7f0eb8c00640 1 -- 172.21.15.107:0/2261457580 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607971c9600 con 0x5607975dbe40 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.037+0000 7f0eb8c00640 10 client.25600 handle_command_reply: tid=0 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.037+0000 7f0ed8da4740 1 client.25600 shutdown 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 msgr2=0x560797579e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797579e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0096d60 tx=0x7f0ec009a390 comp rx=0 tx=0).stop 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 shutdown_connections 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5607975dbe40 0x5607975de230 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56079756c570 0x5607975da340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560797563d60 0x56079757a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 --2- 172.21.15.107:0/2261457580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x560797562ff0 0x560797579e00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0eb8c00640 0 client.25600 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 >> 172.21.15.107:0/2261457580 conn(0x56079752a4b0 msgr2=0x56079755d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 shutdown_connections 2024-03-11T19:54:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 1 -- 172.21.15.107:0/2261457580 wait complete. 2024-03-11T19:54:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.038+0000 7f0ed8da4740 20 client.25600 trim_cache size 0 max 16384 2024-03-11T19:54:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:54:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:54:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:54:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 msgr2=0x7f0eb40a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 0x7f0eb40a41f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005fd0 tx=0x7f0ec8009500 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 msgr2=0x7f0ed019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000e9c0 tx=0x7f0ec000ee80 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 shutdown_connections 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0eb40a1d30 0x7f0eb40a41f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed010d7a0 0x7f0ed0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ed0073ed0 0x7f0ed019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 --2- 172.21.15.107:0/355145009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed00735b0 0x7f0ed019f600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 >> 172.21.15.107:0/355145009 conn(0x7f0ed00fcad0 msgr2=0x7f0ed00fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 shutdown_connections 2024-03-11T19:54:21.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:20 smithi107 ceph-mon[31712]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:21.039+0000 7f0ed4a00640 1 -- 172.21.15.107:0/355145009 wait complete. 2024-03-11T19:54:23.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:22 smithi122 ceph-mon[36397]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:23.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:22 smithi145 ceph-mon[36730]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:23.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:22 smithi107 ceph-mon[31712]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:54:25.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:24 smithi122 ceph-mon[36397]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-11T19:54:25.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:24 smithi145 ceph-mon[36730]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-11T19:54:25.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:24 smithi107 ceph-mon[31712]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-11T19:54:27.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:26 smithi122 ceph-mon[36397]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:27.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:26 smithi145 ceph-mon[36730]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:27.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:26 smithi107 ceph-mon[31712]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:29.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:28 smithi122 ceph-mon[36397]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:29.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:28 smithi145 ceph-mon[36730]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:29.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:28 smithi107 ceph-mon[31712]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:31.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:30 smithi122 ceph-mon[36397]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:31.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:30 smithi145 ceph-mon[36730]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:31.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:30 smithi107 ceph-mon[31712]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:54:33.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:32 smithi122 ceph-mon[36397]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:33.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:32 smithi145 ceph-mon[36730]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:33.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:32 smithi107 ceph-mon[31712]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 -- 172.21.15.107:0/1579234046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f622810b550 msgr2=0x7f622810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 --2- 172.21.15.107:0/1579234046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f622810b550 0x7f622810d9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6220009be0 tx=0x7f622002bf80 comp rx=0 tx=0).stop 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 -- 172.21.15.107:0/1579234046 shutdown_connections 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 --2- 172.21.15.107:0/1579234046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f622810b550 0x7f622810d9f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 --2- 172.21.15.107:0/1579234046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102a50 0x7f622810af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 --2- 172.21.15.107:0/1579234046 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228102130 0x7f6228102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 -- 172.21.15.107:0/1579234046 >> 172.21.15.107:0/1579234046 conn(0x7f62280fb990 msgr2=0x7f62280fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 -- 172.21.15.107:0/1579234046 shutdown_connections 2024-03-11T19:54:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.160+0000 7f6227200640 1 -- 172.21.15.107:0/1579234046 wait complete. 2024-03-11T19:54:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.161+0000 7f6227200640 1 Processor -- start 2024-03-11T19:54:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.161+0000 7f6227200640 1 -- start start 2024-03-11T19:54:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 0x7f62281995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6228102a50 0x7f6228199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819e460 con 0x7f6228102130 2024-03-11T19:54:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819e5d0 con 0x7f6228102a50 2024-03-11T19:54:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6227200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622819e790 con 0x7f622810b550 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6226800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6226800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52510/0 (socket says 172.21.15.107:52510) 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6226800640 1 -- 172.21.15.107:0/4042380189 learned_addr learned my addr 172.21.15.107:0/4042380189 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6225400640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6228102a50 0x7f6228199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6225e00640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 0x7f62281995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6226800640 1 -- 172.21.15.107:0/4042380189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6228102a50 msgr2=0x7f6228199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.162+0000 7f6226800640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6228102a50 0x7f6228199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6226800640 1 -- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 msgr2=0x7f62281995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6226800640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 0x7f62281995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6226800640 1 -- 172.21.15.107:0/4042380189 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009490 con 0x7f622810b550 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6225e00640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 0x7f62281995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6226800640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6220036040 tx=0x7f62200043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f621ea00640 1 -- 172.21.15.107:0/4042380189 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220004870 con 0x7f622810b550 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6220009840 con 0x7f622810b550 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f621ea00640 1 -- 172.21.15.107:0/4042380189 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6220034910 con 0x7f622810b550 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.163+0000 7f621ea00640 1 -- 172.21.15.107:0/4042380189 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622003e7b0 con 0x7f622810b550 2024-03-11T19:54:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.164+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a6110 con 0x7f622810b550 2024-03-11T19:54:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.167+0000 7f621ea00640 1 -- 172.21.15.107:0/4042380189 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f6220034a80 con 0x7f622810b550 2024-03-11T19:54:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.168+0000 7f622fa2e740 1 Processor -- start 2024-03-11T19:54:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.168+0000 7f622fa2e740 1 -- start start 2024-03-11T19:54:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f621ea00640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 0x7f620c0a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 0x5583e41d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 0x5583e41e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e4191620 con 0x5583e41d6d60 2024-03-11T19:54:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e41438c0 con 0x5583e41df570 2024-03-11T19:54:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f622fa2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e4140df0 con 0x5583e41d5ff0 2024-03-11T19:54:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f621ea00640 1 -- 172.21.15.107:0/4042380189 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f62200f0b60 con 0x7f622810b550 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f6225e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f6225e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44414/0 (socket says 172.21.15.107:44414) 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.169+0000 7f6225e00640 1 -- 172.21.15.107:0/986490457 learned_addr learned my addr 172.21.15.107:0/986490457 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 0x5583e41e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6225400640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 0x5583e41d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 -- 172.21.15.107:0/986490457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 msgr2=0x5583e41d6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 0x5583e41d6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 -- 172.21.15.107:0/986490457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 msgr2=0x5583e41d5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6226800640 1 -- 172.21.15.107:0/986490457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e40bdb00 con 0x5583e41df570 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6225400640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 0x5583e41d6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.170+0000 7f6225400640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 0x7f620c0a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.171+0000 7f6226800640 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 0x5583e41e1960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62200ede20 tx=0x7f622002cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44fa8605230099cc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.171+0000 7f6213000640 1 -- 172.21.15.107:0/986490457 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200bd230 con 0x5583e41df570 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.171+0000 7f6213000640 1 -- 172.21.15.107:0/986490457 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62200c1380 con 0x5583e41df570 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.171+0000 7f6213000640 1 -- 172.21.15.107:0/986490457 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200ee250 con 0x5583e41df570 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 -- 172.21.15.107:0/986490457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 msgr2=0x5583e41e1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 0x5583e41e1960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62200ede20 tx=0x7f622002cbe0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f6225400640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 0x7f620c0a41f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6214005f60 tx=0x7f6214005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 -- 172.21.15.107:0/986490457 shutdown_connections 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41df570 0x5583e41e1960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41d5ff0 0x5583e41d6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 --2- 172.21.15.107:0/986490457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 -- 172.21.15.107:0/986490457 >> 172.21.15.107:0/986490457 conn(0x5583e419d4b0 msgr2=0x5583e41d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 -- 172.21.15.107:0/986490457 shutdown_connections 2024-03-11T19:54:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.172+0000 7f622fa2e740 1 -- 172.21.15.107:0/986490457 wait complete. 2024-03-11T19:54:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.173+0000 7f622fa2e740 1 -- start start 2024-03-11T19:54:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.173+0000 7f622fa2e740 1 Processor -- start 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.173+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.173+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f622fa2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e413c5d0 con 0x5583e41d6d60 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f622fa2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e40e1b50 con 0x5583e41d5ff0 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f622fa2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e40e17f0 con 0x5583e41df570 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f622fa2e740 10 client.0 _ll_register_callbacks cb 0x5583e40f5550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36754/0 (socket says 172.21.15.107:36754) 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 -- 172.21.15.107:0/723690405 learned_addr learned my addr 172.21.15.107:0/723690405 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6226800640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225400640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 -- 172.21.15.107:0/723690405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 msgr2=0x5583e41dd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41dd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 -- 172.21.15.107:0/723690405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 msgr2=0x5583e41dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225e00640 1 -- 172.21.15.107:0/723690405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281a6110 con 0x5583e41d5ff0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.174+0000 7f6225400640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f6225e00640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6218013780 tx=0x7f6218013c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2f01dd878df472 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f6207400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011d70 con 0x5583e41d5ff0 2024-03-11T19:54:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f621801b3e0 con 0x5583e41d5ff0 2024-03-11T19:54:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583e410ecd0 con 0x5583e41d5ff0 2024-03-11T19:54:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6218011ed0 con 0x5583e41d5ff0 2024-03-11T19:54:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f622fa2e740 10 client.25604 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.175+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5583e413ac50 con 0x5583e41d5ff0 2024-03-11T19:54:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583e410ecd0 con 0x5583e41d5ff0 2024-03-11T19:54:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f621801c850 con 0x5583e41d5ff0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 10 client.25604 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 20 client.25604 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e41f7bd0 0x5583e41fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 4 client.25604 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.176+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583e4132ce0 con 0x5583e41f7bd0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.177+0000 7f6225400640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e41f7bd0 0x5583e41fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.177+0000 7f6225400640 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e41f7bd0 0x5583e41fa010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387dca7a32f9f428 server_cookie=aa1c87a445aadd16 in_seq=0 out_seq=0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.177+0000 7f6207400640 10 client.25604 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f6207400640 1 -- 172.21.15.107:0/723690405 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583e4132ce0 con 0x5583e41f7bd0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f6207400640 10 client.25604 handle_command_reply: tid=0 2024-03-11T19:54:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 client.25604 shutdown 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 msgr2=0x5583e41dc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6218013780 tx=0x7f6218013c40 comp rx=0 tx=0).stop 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 shutdown_connections 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e41f7bd0 0x5583e41fa010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 --2- 172.21.15.107:0/723690405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41dc5d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 >> 172.21.15.107:0/723690405 conn(0x5583e419d4b0 msgr2=0x5583e41d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f6207400640 0 client.25604 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 shutdown_connections 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.178+0000 7f622fa2e740 1 -- 172.21.15.107:0/723690405 wait complete. 2024-03-11T19:54:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.179+0000 7f622fa2e740 20 client.25604 trim_cache size 0 max 16384 2024-03-11T19:54:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.207+0000 7f622fa2e740 1 Processor -- start 2024-03-11T19:54:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.207+0000 7f622fa2e740 1 -- start start 2024-03-11T19:54:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 0x5583e41ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41e6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e3e84e60 con 0x5583e41d5ff0 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e41cc750 con 0x5583e41d6d60 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f622fa2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e41d5510 con 0x5583e41df570 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6225400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6225400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36762/0 (socket says 172.21.15.107:36762) 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6225e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 0x5583e41ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6225400640 1 -- 172.21.15.107:0/2422938029 learned_addr learned my addr 172.21.15.107:0/2422938029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6226800640 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.208+0000 7f6225400640 1 -- 172.21.15.107:0/2422938029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 msgr2=0x5583e41e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225400640 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41e6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225400640 1 -- 172.21.15.107:0/2422938029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 msgr2=0x5583e41ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225400640 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 0x5583e41ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225400640 1 -- 172.21.15.107:0/2422938029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e40d7530 con 0x5583e41d6d60 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225e00640 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 0x5583e41ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6225400640 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5583e41d74c0 tx=0x7f6214009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6e63a612ecc63 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6212600640 1 -- 172.21.15.107:0/2422938029 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214024070 con 0x5583e41d6d60 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6212600640 1 -- 172.21.15.107:0/2422938029 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f621400dd00 con 0x5583e41d6d60 2024-03-11T19:54:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.209+0000 7f6212600640 1 -- 172.21.15.107:0/2422938029 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214005280 con 0x5583e41d6d60 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 -- 172.21.15.107:0/2422938029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 msgr2=0x5583e41e6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5583e41d74c0 tx=0x7f6214009800 comp rx=0 tx=0).stop 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 -- 172.21.15.107:0/2422938029 shutdown_connections 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d6d60 0x5583e41e6110 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 --2- 172.21.15.107:0/2422938029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d5ff0 0x5583e41ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 -- 172.21.15.107:0/2422938029 >> 172.21.15.107:0/2422938029 conn(0x5583e419d4b0 msgr2=0x5583e41d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 -- 172.21.15.107:0/2422938029 shutdown_connections 2024-03-11T19:54:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.210+0000 7f622fa2e740 1 -- 172.21.15.107:0/2422938029 wait complete. 2024-03-11T19:54:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.211+0000 7f622fa2e740 1 -- start start 2024-03-11T19:54:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.211+0000 7f622fa2e740 1 Processor -- start 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e41e6a40 con 0x5583e41d6d60 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e424e130 con 0x5583e41d5ff0 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e424e270 con 0x5583e41df570 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f622fa2e740 10 client.0 _ll_register_callbacks cb 0x7f62180097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f6225e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f6225e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36772/0 (socket says 172.21.15.107:36772) 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f6225e00640 1 -- 172.21.15.107:0/1705805963 learned_addr learned my addr 172.21.15.107:0/1705805963 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.212+0000 7f6226800640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225400640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 -- 172.21.15.107:0/1705805963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 msgr2=0x5583e41d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 -- 172.21.15.107:0/1705805963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 msgr2=0x5583e41d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 -- 172.21.15.107:0/1705805963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62140124b0 con 0x5583e41d5ff0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225400640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6225e00640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218009530 tx=0x7f62180058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fab46b64cef806 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6207400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011d70 con 0x5583e41d5ff0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6218000e40 con 0x5583e41d5ff0 2024-03-11T19:54:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.213+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583e4161d30 con 0x5583e41d5ff0 2024-03-11T19:54:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.214+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6218011ed0 con 0x5583e41d5ff0 2024-03-11T19:54:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.214+0000 7f622fa2e740 10 client.25610 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.214+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5583e40fd2c0 con 0x5583e41d5ff0 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.214+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583e4161d30 con 0x5583e41d5ff0 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.214+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6218033070 con 0x5583e41d5ff0 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 10 client.25610 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 20 client.25610 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e424f970 0x5583e4251d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 4 client.25610 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583e3e3c600 con 0x5583e424f970 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.215+0000 7f6225400640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e424f970 0x5583e4251d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.216+0000 7f6207400640 10 client.25610 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.216+0000 7f6225400640 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e424f970 0x5583e4251d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab0adcda0030494 server_cookie=56f55da4b1d9e07e in_seq=0 out_seq=0 2024-03-11T19:54:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.220+0000 7f6207400640 1 -- 172.21.15.107:0/1705805963 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583e3e3c600 con 0x5583e424f970 2024-03-11T19:54:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.220+0000 7f6207400640 10 client.25610 handle_command_reply: tid=0 2024-03-11T19:54:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.220+0000 7f622fa2e740 1 client.25610 shutdown 2024-03-11T19:54:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 msgr2=0x5583e41d9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6218009530 tx=0x7f62180058d0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 shutdown_connections 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5583e424f970 0x5583e4251d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5583e41df570 0x5583e41d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5583e41d6d60 0x5583e41d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 --2- 172.21.15.107:0/1705805963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e41d5ff0 0x5583e41d9100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 >> 172.21.15.107:0/1705805963 conn(0x5583e419d4b0 msgr2=0x5583e41cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f6207400640 0 client.25610 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 shutdown_connections 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 1 -- 172.21.15.107:0/1705805963 wait complete. 2024-03-11T19:54:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.221+0000 7f622fa2e740 20 client.25610 trim_cache size 0 max 16384 2024-03-11T19:54:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:54:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:54:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:54:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 msgr2=0x7f620c0a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 0x7f620c0a41f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6214005f60 tx=0x7f6214005f90 comp rx=0 tx=0).stop 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 msgr2=0x7f622819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6220036040 tx=0x7f62200043b0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 shutdown_connections 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f620c0a1d30 0x7f620c0a41f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6228102a50 0x7f6228199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f622810b550 0x7f622819de60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 --2- 172.21.15.107:0/4042380189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6228102130 0x7f62281995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.224+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 >> 172.21.15.107:0/4042380189 conn(0x7f62280fb990 msgr2=0x7f622810cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.225+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 shutdown_connections 2024-03-11T19:54:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:34.225+0000 7f6227200640 1 -- 172.21.15.107:0/4042380189 wait complete. 2024-03-11T19:54:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:34 smithi122 ceph-mon[36397]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:35.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:34 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:34 smithi145 ceph-mon[36730]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:35.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:34 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:35.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:34 smithi107 ceph-mon[31712]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:54:35.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:34 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:35 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:35 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:36.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:35 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:54:36.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:35 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:36.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:35 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:36.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:35 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:54:36.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:35 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:36.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:35 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:36.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:35 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:54:37.013 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:36 smithi122 ceph-mon[36397]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:54:37.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:36 smithi145 ceph-mon[36730]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:54:37.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:36 smithi107 ceph-mon[31712]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-11T19:54:37.567 INFO:tasks.workunit.client.0.smithi107.stderr:Updating files: 65% (8024/12344) Updating files: 66% (8148/12344) Updating files: 67% (8271/12344) Updating files: 68% (8394/12344) Updating files: 69% (8518/12344) Updating files: 70% (8641/12344) Updating files: 71% (8765/12344) Updating files: 72% (8888/12344) Updating files: 73% (9012/12344) Updating files: 74% (9135/12344) Updating files: 75% (9258/12344) Updating files: 76% (9382/12344) Updating files: 77% (9505/12344) Updating files: 78% (9629/12344) Updating files: 79% (9752/12344) Updating files: 80% (9876/12344) Updating files: 81% (9999/12344) Updating files: 82% (10123/12344) Updating files: 83% (10246/12344) Updating files: 84% (10369/12344) Updating files: 85% (10493/12344) Updating files: 86% (10616/12344) Updating files: 87% (10740/12344) Updating files: 88% (10863/12344) Updating files: 89% (10987/12344) Updating files: 90% (11110/12344) Updating files: 91% (11234/12344) Updating files: 92% (11357/12344) Updating files: 93% (11480/12344) Updating files: 94% (11604/12344) Updating files: 95% (11727/12344) Updating files: 96% (11851/12344) Updating files: 97% (11974/12344) Updating files: 98% (12098/12344) Updating files: 99% (12221/12344) Updating files: 100% (12344/12344) Updating files: 100% (12344/12344), done. 2024-03-11T19:54:38.121 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'. 2024-03-11T19:54:38.121 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.121 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-11T19:54:38.121 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-11T19:54:38.121 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-11T19:54:38.122 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at b6a4997be2b Merge pull request #56060 from rhcs-dashboard/wip-64808-squid 2024-03-11T19:54:38.128 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-11T19:54:38.236 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-11T19:54:38.243 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T19:54:38.243 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-11T19:54:39.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:38 smithi122 ceph-mon[36397]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:54:39.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:38 smithi145 ceph-mon[36730]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:54:39.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:38 smithi107 ceph-mon[31712]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-11T19:54:39.208 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-11T19:54:39.358 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-11T19:54:39.444 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T19:54:39.451 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T19:54:39.451 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-11T19:54:39.531 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T19:54:39.541 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T19:54:39.541 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-11T19:54:39.592 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-11T19:54:39.595 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-11T19:54:39.595 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-11T19:54:39.714 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p fsstress 2024-03-11T19:54:39.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd fsstress 2024-03-11T19:54:39.715 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:54:39.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-11T19:54:41.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:40 smithi122 ceph-mon[36397]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:54:41.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:40 smithi145 ceph-mon[36730]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:54:41.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:40 smithi107 ceph-mon[31712]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-11T19:54:43.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:42 smithi122 ceph-mon[36397]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:54:43.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:42 smithi145 ceph-mon[36730]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:54:43.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:42 smithi107 ceph-mon[31712]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-11T19:54:44.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:43 smithi122 ceph-mon[36397]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:54:44.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:43 smithi145 ceph-mon[36730]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:54:44.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:43 smithi107 ceph-mon[31712]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:54:46.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:46 smithi122 ceph-mon[36397]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:46.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:46 smithi145 ceph-mon[36730]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:46.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:46 smithi107 ceph-mon[31712]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 -- 172.21.15.107:0/2285515439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a410d760 msgr2=0x7f51a410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 --2- 172.21.15.107:0/2285515439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a410d760 0x7f51a410db40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5194009cd0 tx=0x7f519402c0e0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 -- 172.21.15.107:0/2285515439 shutdown_connections 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 --2- 172.21.15.107:0/2285515439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 0x7f51a4114620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 --2- 172.21.15.107:0/2285515439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a4107750 0x7f51a4107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 --2- 172.21.15.107:0/2285515439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a410d760 0x7f51a410db40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 -- 172.21.15.107:0/2285515439 >> 172.21.15.107:0/2285515439 conn(0x7f51a40756f0 msgr2=0x7f51a4077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 -- 172.21.15.107:0/2285515439 shutdown_connections 2024-03-11T19:54:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.371+0000 7f51ab000640 1 -- 172.21.15.107:0/2285515439 wait complete. 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.372+0000 7f51ab000640 1 Processor -- start 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 -- start start 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 0x7f51a419e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41a2c00 con 0x7f51a4107750 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41a2d70 con 0x7f51a4108110 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51ab000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41a2f30 con 0x7f51a410d760 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51a9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51a9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38436/0 (socket says 172.21.15.107:38436) 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51a9000640 1 -- 172.21.15.107:0/715623119 learned_addr learned my addr 172.21.15.107:0/715623119 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36106/0 (socket says 172.21.15.107:36106) 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.373+0000 7f519b400640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 0x7f51a419e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 -- 172.21.15.107:0/715623119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 msgr2=0x7f51a41a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 -- 172.21.15.107:0/715623119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 msgr2=0x7f51a419e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 0x7f51a419e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 -- 172.21.15.107:0/715623119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009900 con 0x7f51a4107750 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a9000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a3e00640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f519402c6f0 tx=0x7f51940325d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a1600640 1 -- 172.21.15.107:0/715623119 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940328b0 con 0x7f51a4107750 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a1600640 1 -- 172.21.15.107:0/715623119 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5194003e50 con 0x7f51a4107750 2024-03-11T19:54:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51a1600640 1 -- 172.21.15.107:0/715623119 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194004860 con 0x7f51a4107750 2024-03-11T19:54:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a41a31b0 con 0x7f51a4107750 2024-03-11T19:54:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.374+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a41aa8b0 con 0x7f51a4107750 2024-03-11T19:54:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.377+0000 7f51a1600640 1 -- 172.21.15.107:0/715623119 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 136513+0+0 (secure 0 0 0) 0x7f519400a040 con 0x7f51a4107750 2024-03-11T19:54:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 Processor -- start 2024-03-11T19:54:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 -- start start 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d51870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d5d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7d0d620 con 0x55f6c7d51db0 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7cbf8c0 con 0x55f6c7d52b20 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51ade34740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7cbcdf0 con 0x55f6c7d5b330 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.378+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36118/0 (socket says 172.21.15.107:36118) 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 -- 172.21.15.107:0/395759358 learned_addr learned my addr 172.21.15.107:0/395759358 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a9000640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d51870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f519b400640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d5d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 -- 172.21.15.107:0/395759358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 msgr2=0x55f6c7d5d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d5d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 -- 172.21.15.107:0/395759358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 msgr2=0x55f6c7d51870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d51870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 -- 172.21.15.107:0/395759358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6c7c39b00 con 0x55f6c7d51db0 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a9000640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d51870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a1600640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 0x7f51880a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a1600640 1 -- 172.21.15.107:0/715623119 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f51940f0560 con 0x7f51a4107750 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a3e00640 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51940ed8b0 tx=0x7f51940f65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ca9c37ba839142 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.379+0000 7f51a9000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 0x7f51880a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f5199600640 1 -- 172.21.15.107:0/395759358 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032a10 con 0x55f6c7d51db0 2024-03-11T19:54:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f5199600640 1 -- 172.21.15.107:0/395759358 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51940bca90 con 0x55f6c7d51db0 2024-03-11T19:54:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f5199600640 1 -- 172.21.15.107:0/395759358 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940bd3c0 con 0x55f6c7d51db0 2024-03-11T19:54:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f51ade34740 1 -- 172.21.15.107:0/395759358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 msgr2=0x55f6c7d52230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f51a9000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 0x7f51880a42a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f519c009640 tx=0x7f519c005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.380+0000 7f51ade34740 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51940ed8b0 tx=0x7f51940f65a0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 -- 172.21.15.107:0/395759358 shutdown_connections 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d5d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d51db0 0x55f6c7d52230 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 --2- 172.21.15.107:0/395759358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d51870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 -- 172.21.15.107:0/395759358 >> 172.21.15.107:0/395759358 conn(0x55f6c7d194b0 msgr2=0x55f6c7d4caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 -- 172.21.15.107:0/395759358 shutdown_connections 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 -- 172.21.15.107:0/395759358 wait complete. 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 -- start start 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.381+0000 7f51ade34740 1 Processor -- start 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7cb85d0 con 0x55f6c7d5b330 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7c5db50 con 0x55f6c7d51db0 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7c5d7f0 con 0x55f6c7d52b20 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51ade34740 10 client.0 _ll_register_callbacks cb 0x55f6c7c71550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36134/0 (socket says 172.21.15.107:36134) 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51a9000640 1 -- 172.21.15.107:0/3558887818 learned_addr learned my addr 172.21.15.107:0/3558887818 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f519b400640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.382+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 -- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 msgr2=0x55f6c7d58310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d58310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 -- 172.21.15.107:0/3558887818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 msgr2=0x55f6c7d58d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 -- 172.21.15.107:0/3558887818 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a41aa8b0 con 0x55f6c7d52b20 2024-03-11T19:54:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f51a9000640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f519b400640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51a4077d20 tx=0x7f518c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2eb3acf5732f3f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f5192000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c017070 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f518c00f920 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.383+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6c7c8acd0 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518c00fae0 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 10 client.25630 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6c7cb6c50 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6c7c8acd0 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f518c010d10 con 0x55f6c7d52b20 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 10 client.25630 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 20 client.25630 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7d6b1e0 0x55f6c7d6d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 4 client.25630 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.384+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6c7caece0 con 0x55f6c7d6b1e0 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.385+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7d6b1e0 0x55f6c7d6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.385+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7d6b1e0 0x55f6c7d6d600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787d7fb20ca0c34a server_cookie=fb53ede49902c722 in_seq=0 out_seq=0 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.385+0000 7f5192000640 10 client.25630 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f5192000640 1 -- 172.21.15.107:0/3558887818 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f6c7caece0 con 0x55f6c7d6b1e0 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f5192000640 10 client.25630 handle_command_reply: tid=0 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 client.25630 shutdown 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 msgr2=0x55f6c7d58850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51a4077d20 tx=0x7f518c012b60 comp rx=0 tx=0).stop 2024-03-11T19:54:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 shutdown_connections 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7d6b1e0 0x55f6c7d6d600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7d58d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58850 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 --2- 172.21.15.107:0/3558887818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f5192000640 0 client.25630 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 >> 172.21.15.107:0/3558887818 conn(0x55f6c7d194b0 msgr2=0x55f6c7d4e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 shutdown_connections 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 1 -- 172.21.15.107:0/3558887818 wait complete. 2024-03-11T19:54:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.386+0000 7f51ade34740 20 client.25630 trim_cache size 0 max 16384 2024-03-11T19:54:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 Processor -- start 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 -- start start 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d5aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7a00e60 con 0x55f6c7d52b20 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7d4d8a0 con 0x55f6c7d51db0 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51ade34740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7d51080 con 0x55f6c7d5b330 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51a9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d5aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51a9000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38466/0 (socket says 172.21.15.107:38466) 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f51a9000640 1 -- 172.21.15.107:0/3710874319 learned_addr learned my addr 172.21.15.107:0/3710874319 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.401+0000 7f519b400640 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 -- 172.21.15.107:0/3710874319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 msgr2=0x55f6c7d56be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 -- 172.21.15.107:0/3710874319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d52b20 msgr2=0x55f6c7d58bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 -- 172.21.15.107:0/3710874319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6c7c53530 con 0x55f6c7d51db0 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a9000640 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f51a3e00640 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d5aa40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5194041000 tx=0x7f519402c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27447951e3e0b348 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f5198c00640 1 -- 172.21.15.107:0/3710874319 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032a10 con 0x55f6c7d51db0 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f5198c00640 1 -- 172.21.15.107:0/3710874319 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51940bc800 con 0x55f6c7d51db0 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.402+0000 7f5198c00640 1 -- 172.21.15.107:0/3710874319 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940bccc0 con 0x55f6c7d51db0 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 -- 172.21.15.107:0/3710874319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 msgr2=0x55f6c7d5aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d5aa40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5194041000 tx=0x7f519402c0e0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 -- 172.21.15.107:0/3710874319 shutdown_connections 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d5b330 0x55f6c7d56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d52b20 0x55f6c7d58bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 --2- 172.21.15.107:0/3710874319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d51db0 0x55f6c7d5aa40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 -- 172.21.15.107:0/3710874319 >> 172.21.15.107:0/3710874319 conn(0x55f6c7d194b0 msgr2=0x55f6c7d535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 -- 172.21.15.107:0/3710874319 shutdown_connections 2024-03-11T19:54:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.403+0000 7f51ade34740 1 -- 172.21.15.107:0/3710874319 wait complete. 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 -- start start 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 Processor -- start 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d51db0 0x55f6c7d60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7dba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7dbabf0 con 0x55f6c7d5b330 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7dbad30 con 0x55f6c7d52b20 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6c7dbaef0 con 0x55f6c7d51db0 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51ade34740 10 client.0 _ll_register_callbacks cb 0x7f518c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f519b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f51a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7dba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.404+0000 7f519b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:57856/0 (socket says 172.21.15.107:57856) 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 -- 172.21.15.107:0/309306930 learned_addr learned my addr 172.21.15.107:0/309306930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:54:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f51a3e00640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d51db0 0x55f6c7d60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 -- 172.21.15.107:0/309306930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d51db0 msgr2=0x55f6c7d60170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d51db0 0x55f6c7d60170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 -- 172.21.15.107:0/309306930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 msgr2=0x55f6c7dba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7dba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 -- 172.21.15.107:0/309306930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c012e50 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f519b400640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55f6c7d598e0 tx=0x7f518c00ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0df0b2763318499 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f5192000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c017070 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6c7cddd30 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.405+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f518c005520 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518c0056e0 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 10 client.25628 fetch_fsmap learned FSMap version 24 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6c7c39b00 con 0x55f6c7d52b20 2024-03-11T19:54:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f6c7cddd30 con 0x55f6c7d52b20 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f518c0037b0 con 0x55f6c7d52b20 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 10 client.25628 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 20 client.25628 populate_metadata read hostname 'smithi107' 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7dc27e0 0x55f6c7dc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 4 client.25628 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:54:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6c79b8600 con 0x55f6c7dc27e0 2024-03-11T19:54:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.406+0000 7f51a3e00640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7dc27e0 0x55f6c7dc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.407+0000 7f51a3e00640 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7dc27e0 0x55f6c7dc4c00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21b9068f6e0f1c5 server_cookie=2e875197b1decf51 in_seq=0 out_seq=0 2024-03-11T19:54:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.407+0000 7f5192000640 10 client.25628 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f5192000640 1 -- 172.21.15.107:0/309306930 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6c79b8600 con 0x55f6c7dc27e0 2024-03-11T19:54:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f5192000640 10 client.25628 handle_command_reply: tid=0 2024-03-11T19:54:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 client.25628 shutdown 2024-03-11T19:54:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 msgr2=0x55f6c7d606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55f6c7d598e0 tx=0x7f518c00ff50 comp rx=0 tx=0).stop 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 shutdown_connections 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f6c7dc27e0 0x55f6c7dc4c00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f6c7d52b20 0x55f6c7d606b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6c7d5b330 0x55f6c7dba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f5192000640 0 client.25628 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.420+0000 7f51ade34740 1 --2- 172.21.15.107:0/309306930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f6c7d51db0 0x55f6c7d60170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.421+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 >> 172.21.15.107:0/309306930 conn(0x55f6c7d194b0 msgr2=0x55f6c7d56ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.421+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 shutdown_connections 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.421+0000 7f51ade34740 1 -- 172.21.15.107:0/309306930 wait complete. 2024-03-11T19:54:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.421+0000 7f51ade34740 20 client.25628 trim_cache size 0 max 16384 2024-03-11T19:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:54:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 msgr2=0x7f51880a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 0x7f51880a42a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f519c009640 tx=0x7f519c005f90 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 msgr2=0x7f51a419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f519402c6f0 tx=0x7f51940325d0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 shutdown_connections 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f51880a1de0 0x7f51880a42a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4108110 0x7f51a419e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f51a410d760 0x7f51a41a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 --2- 172.21.15.107:0/715623119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a4107750 0x7f51a419dd90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 >> 172.21.15.107:0/715623119 conn(0x7f51a40756f0 msgr2=0x7f51a4075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 shutdown_connections 2024-03-11T19:54:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:54:47.422+0000 7f51ab000640 1 -- 172.21.15.107:0/715623119 wait complete. 2024-03-11T19:54:48.762 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:48 smithi122 ceph-mon[36397]: pgmap v44: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 136 KiB/s wr, 8 op/s 2024-03-11T19:54:48.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:48 smithi145 ceph-mon[36730]: pgmap v44: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 136 KiB/s wr, 8 op/s 2024-03-11T19:54:48.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:48 smithi107 ceph-mon[31712]: pgmap v44: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 136 KiB/s wr, 8 op/s 2024-03-11T19:54:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:50 smithi145 ceph-mon[36730]: pgmap v45: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 135 KiB/s wr, 5 op/s 2024-03-11T19:54:50.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:50 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:50.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:50 smithi107 ceph-mon[31712]: pgmap v45: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 135 KiB/s wr, 5 op/s 2024-03-11T19:54:50.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:50 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:51.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:50 smithi122 ceph-mon[36397]: pgmap v45: 129 pgs: 129 active+clean; 2.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 135 KiB/s wr, 5 op/s 2024-03-11T19:54:51.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:50 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:52.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:52 smithi145 ceph-mon[36730]: pgmap v46: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:52.793 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:52 smithi107 ceph-mon[31712]: pgmap v46: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:53.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:52 smithi122 ceph-mon[36397]: pgmap v46: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:54.787 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:54 smithi145 ceph-mon[36730]: pgmap v47: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:54.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:54 smithi107 ceph-mon[31712]: pgmap v47: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:55.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:54 smithi122 ceph-mon[36397]: pgmap v47: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 140 KiB/s wr, 10 op/s 2024-03-11T19:54:56.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:56 smithi107 ceph-mon[31712]: pgmap v48: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 140 KiB/s wr, 9 op/s 2024-03-11T19:54:57.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:56 smithi122 ceph-mon[36397]: pgmap v48: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 140 KiB/s wr, 9 op/s 2024-03-11T19:54:57.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:56 smithi145 ceph-mon[36730]: pgmap v48: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 140 KiB/s wr, 9 op/s 2024-03-11T19:54:59.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:58 smithi122 ceph-mon[36397]: pgmap v49: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 142 KiB/s wr, 13 op/s 2024-03-11T19:54:59.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:54:58 smithi122 ceph-mon[36397]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-11T19:54:59.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:58 smithi145 ceph-mon[36730]: pgmap v49: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 142 KiB/s wr, 13 op/s 2024-03-11T19:54:59.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:54:58 smithi145 ceph-mon[36730]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-11T19:54:59.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:58 smithi107 ceph-mon[31712]: pgmap v49: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 142 KiB/s wr, 13 op/s 2024-03-11T19:54:59.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:54:58 smithi107 ceph-mon[31712]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-11T19:55:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 -- 172.21.15.107:0/2785186883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90103f80 msgr2=0x7f2a90107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 --2- 172.21.15.107:0/2785186883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90103f80 0x7f2a90107fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009930 tx=0x7f2a7802bc20 comp rx=0 tx=0).stop 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 -- 172.21.15.107:0/2785186883 shutdown_connections 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 --2- 172.21.15.107:0/2785186883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901086a0 0x7f2a90110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 --2- 172.21.15.107:0/2785186883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90103f80 0x7f2a90107fc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.565+0000 7f2a97600640 1 --2- 172.21.15.107:0/2785186883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901035d0 0x7f2a901039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.566+0000 7f2a97600640 1 -- 172.21.15.107:0/2785186883 >> 172.21.15.107:0/2785186883 conn(0x7f2a900fee30 msgr2=0x7f2a90101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.566+0000 7f2a97600640 1 -- 172.21.15.107:0/2785186883 shutdown_connections 2024-03-11T19:55:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.566+0000 7f2a97600640 1 -- 172.21.15.107:0/2785186883 wait complete. 2024-03-11T19:55:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.567+0000 7f2a97600640 1 Processor -- start 2024-03-11T19:55:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.567+0000 7f2a97600640 1 -- start start 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.567+0000 7f2a97600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.567+0000 7f2a97600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901086a0 0x7f2a9019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.567+0000 7f2a97600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90199770 0x7f2a90199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a97600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9019a240 con 0x7f2a90199770 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a97600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9019a3b0 con 0x7f2a901086a0 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a97600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9019a570 con 0x7f2a901035d0 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52884/0 (socket says 172.21.15.107:52884) 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 -- 172.21.15.107:0/1794822480 learned_addr learned my addr 172.21.15.107:0/1794822480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a95600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90199770 0x7f2a90199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a87e00640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901086a0 0x7f2a9019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 -- 172.21.15.107:0/1794822480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901086a0 msgr2=0x7f2a9019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901086a0 0x7f2a9019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 -- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90199770 msgr2=0x7f2a90199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90199770 0x7f2a90199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.568+0000 7f2a94c00640 1 -- 172.21.15.107:0/1794822480 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78009560 con 0x7f2a901035d0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a94c00640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000d950 tx=0x7f2a8000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a8e000640 1 -- 172.21.15.107:0/1794822480 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80009a30 con 0x7f2a901035d0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a8e000640 1 -- 172.21.15.107:0/1794822480 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a80009b90 con 0x7f2a901035d0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a901a6510 con 0x7f2a901035d0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a8e000640 1 -- 172.21.15.107:0/1794822480 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004a00 con 0x7f2a901035d0 2024-03-11T19:55:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.569+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901a6a40 con 0x7f2a901035d0 2024-03-11T19:55:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.572+0000 7f2a8e000640 1 -- 172.21.15.107:0/1794822480 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2a80009d00 con 0x7f2a901035d0 2024-03-11T19:55:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.572+0000 7f2a9a40a740 1 Processor -- start 2024-03-11T19:55:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.572+0000 7f2a9a40a740 1 -- start start 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x56415471b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415471c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154727720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641546d7620 con 0x564154725330 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641546898c0 con 0x56415471cb20 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564154686df0 con 0x56415471bdb0 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a94c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415471c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154727720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a94c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415471c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:52886/0 (socket says 172.21.15.107:52886) 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a94c00640 1 -- 172.21.15.107:0/2845106732 learned_addr learned my addr 172.21.15.107:0/2845106732 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a95600640 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x56415471b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.573+0000 7f2a8e000640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 0x7f2a740a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 -- 172.21.15.107:0/2845106732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 msgr2=0x56415471c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415471c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 -- 172.21.15.107:0/2845106732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 msgr2=0x56415471b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x56415471b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 -- 172.21.15.107:0/2845106732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564154603b00 con 0x564154725330 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a8e000640 1 -- 172.21.15.107:0/1794822480 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2a800cd500 con 0x7f2a901035d0 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a95600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 0x7f2a740a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a87e00640 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154727720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009230 tx=0x7f2a78005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aee7a6b1195ea45 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.574+0000 7f2a85c00640 1 -- 172.21.15.107:0/2845106732 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7802dea0 con 0x564154725330 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a85c00640 1 -- 172.21.15.107:0/2845106732 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a78004880 con 0x564154725330 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a85c00640 1 -- 172.21.15.107:0/2845106732 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78004ff0 con 0x564154725330 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 -- 172.21.15.107:0/2845106732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 msgr2=0x564154727720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a95600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 0x7f2a740a4180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005c40 tx=0x7f2a88004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154727720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009230 tx=0x7f2a78005be0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 -- 172.21.15.107:0/2845106732 shutdown_connections 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154727720 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415471c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/2845106732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x56415471b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 -- 172.21.15.107:0/2845106732 >> 172.21.15.107:0/2845106732 conn(0x5641546e34b0 msgr2=0x564154716aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 -- 172.21.15.107:0/2845106732 shutdown_connections 2024-03-11T19:55:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.575+0000 7f2a9a40a740 1 -- 172.21.15.107:0/2845106732 wait complete. 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.576+0000 7f2a9a40a740 1 -- start start 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.576+0000 7f2a9a40a740 1 Processor -- start 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x5641547222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x564154722820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641546825d0 con 0x564154725330 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564154627b50 con 0x56415471cb20 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641546277f0 con 0x56415471bdb0 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a9a40a740 10 client.0 _ll_register_callbacks cb 0x56415463b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48222/0 (socket says 172.21.15.107:48222) 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 -- 172.21.15.107:0/3008425763 learned_addr learned my addr 172.21.15.107:0/3008425763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a87e00640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x564154722820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a94c00640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x5641547222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 -- 172.21.15.107:0/3008425763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 msgr2=0x5641547222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x5641547222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 -- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 msgr2=0x564154722820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x564154722820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.577+0000 7f2a95600640 1 -- 172.21.15.107:0/3008425763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a90110a80 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a95600640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800faf0 tx=0x7f2a88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d6aba13de19b4a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a7e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88022070 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2a8801d5f0 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564154654cd0 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a8801d7b0 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a9a40a740 10 client.15519 fetch_fsmap learned FSMap version 25 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.578+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564154680c50 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564154654cd0 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2a88026ac0 con 0x564154725330 2024-03-11T19:55:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 10 client.15519 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:55:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 20 client.15519 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564154735310 0x564154737730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 4 client.15519 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:55:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564154678ce0 con 0x564154735310 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.579+0000 7f2a94c00640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564154735310 0x564154737730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.580+0000 7f2a94c00640 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564154735310 0x564154737730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a177100a19a273b4 server_cookie=5505e8c999d5c48e in_seq=0 out_seq=0 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.580+0000 7f2a7e000640 10 client.15519 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a7e000640 1 -- 172.21.15.107:0/3008425763 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564154678ce0 con 0x564154735310 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a7e000640 10 client.15519 handle_command_reply: tid=0 2024-03-11T19:55:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 client.15519 shutdown 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 msgr2=0x564154722d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800faf0 tx=0x7f2a88004ab0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 shutdown_connections 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564154735310 0x564154737730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56415471cb20 0x564154722820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564154725330 0x564154722d60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3008425763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x5641547222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 >> 172.21.15.107:0/3008425763 conn(0x5641546e34b0 msgr2=0x5641547187f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a7e000640 0 client.15519 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 shutdown_connections 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3008425763 wait complete. 2024-03-11T19:55:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.581+0000 7f2a9a40a740 20 client.15519 trim_cache size 0 max 16384 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.595+0000 7f2a9a40a740 1 Processor -- start 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.595+0000 7f2a9a40a740 1 -- start start 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x564154724a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415472c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641543cae60 con 0x56415471cb20 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564154731d80 con 0x564154725330 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564154717aa0 con 0x56415471bdb0 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48234/0 (socket says 172.21.15.107:48234) 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 -- 172.21.15.107:0/3473283883 learned_addr learned my addr 172.21.15.107:0/3473283883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a94c00640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x564154724a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a95600640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415472c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 -- 172.21.15.107:0/3473283883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 msgr2=0x564154724a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x564154724a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 -- 172.21.15.107:0/3473283883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 msgr2=0x56415472c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.596+0000 7f2a87e00640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415472c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a87e00640 1 -- 172.21.15.107:0/3473283883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56415461d530 con 0x56415471cb20 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a95600640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415472c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a94c00640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x564154724a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a87e00640 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009230 tx=0x7f2a780044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbab3f1f20d0cf6c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a85200640 1 -- 172.21.15.107:0/3473283883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78004520 con 0x56415471cb20 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a85200640 1 -- 172.21.15.107:0/3473283883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a7802ce90 con 0x56415471cb20 2024-03-11T19:55:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.597+0000 7f2a85200640 1 -- 172.21.15.107:0/3473283883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780055f0 con 0x56415471cb20 2024-03-11T19:55:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3473283883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 msgr2=0x564154722b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009230 tx=0x7f2a780044b0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3473283883 shutdown_connections 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415472c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x564154722b80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/3473283883 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x564154724a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3473283883 >> 172.21.15.107:0/3473283883 conn(0x5641546e34b0 msgr2=0x5641547165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3473283883 shutdown_connections 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- 172.21.15.107:0/3473283883 wait complete. 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 -- start start 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.598+0000 7f2a9a40a740 1 Processor -- start 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415472c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415471e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641547203b0 con 0x56415471cb20 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56415471ea90 con 0x564154725330 2024-03-11T19:55:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56415471ec20 con 0x56415471bdb0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a9a40a740 10 client.0 _ll_register_callbacks cb 0x7f2a88009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48238/0 (socket says 172.21.15.107:48238) 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 -- 172.21.15.107:0/4102889016 learned_addr learned my addr 172.21.15.107:0/4102889016 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a94c00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415472c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a95600640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415471e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 -- 172.21.15.107:0/4102889016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 msgr2=0x56415472c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415472c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 -- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 msgr2=0x56415471e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.599+0000 7f2a87e00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415471e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a87e00640 1 -- 172.21.15.107:0/4102889016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901a6a40 con 0x56415471cb20 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a87e00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a7802c160 tx=0x7f2a7802bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4370071b52062a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a7e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78057030 con 0x56415471cb20 2024-03-11T19:55:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2a78058070 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641546a7d30 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a78004520 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a9a40a740 10 client.15525 fetch_fsmap learned FSMap version 25 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.600+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564154603b00 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5641546a7d30 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f2a7803c050 con 0x56415471cb20 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 10 client.15525 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 20 client.15525 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56415478c390 0x56415478e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 4 client.15525 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564154382600 con 0x56415478c390 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.601+0000 7f2a94c00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56415478c390 0x56415478e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.602+0000 7f2a94c00640 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56415478c390 0x56415478e7b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7efa8d00eacaa1c server_cookie=70a959a839d69e5e in_seq=0 out_seq=0 2024-03-11T19:55:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.602+0000 7f2a7e000640 10 client.15525 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a7e000640 1 -- 172.21.15.107:0/4102889016 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564154382600 con 0x56415478c390 2024-03-11T19:55:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a7e000640 10 client.15525 handle_command_reply: tid=0 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 client.15525 shutdown 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 msgr2=0x56415472c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a7802c160 tx=0x7f2a7802bc20 comp rx=0 tx=0).stop 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 shutdown_connections 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56415478c390 0x56415478e7b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564154725330 0x56415471e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56415471cb20 0x56415472c670 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a7e000640 0 client.15525 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 --2- 172.21.15.107:0/4102889016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56415471bdb0 0x56415472c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 >> 172.21.15.107:0/4102889016 conn(0x5641546e34b0 msgr2=0x564154720ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 shutdown_connections 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 1 -- 172.21.15.107:0/4102889016 wait complete. 2024-03-11T19:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.612+0000 7f2a9a40a740 20 client.15525 trim_cache size 0 max 16384 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 msgr2=0x7f2a740a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 0x7f2a740a4180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005c40 tx=0x7f2a88004a40 comp rx=0 tx=0).stop 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 msgr2=0x7f2a9019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000d950 tx=0x7f2a8000de10 comp rx=0 tx=0).stop 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 shutdown_connections 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2a740a1cc0 0x7f2a740a4180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a901086a0 0x7f2a9019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a90199770 0x7f2a90199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 --2- 172.21.15.107:0/1794822480 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a901035d0 0x7f2a9019f6b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 >> 172.21.15.107:0/1794822480 conn(0x7f2a900fee30 msgr2=0x7f2a900ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.613+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 shutdown_connections 2024-03-11T19:55:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:00.614+0000 7f2a97600640 1 -- 172.21.15.107:0/1794822480 wait complete. 2024-03-11T19:55:01.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:00 smithi122 ceph-mon[36397]: pgmap v50: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-11T19:55:01.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:00 smithi145 ceph-mon[36730]: pgmap v50: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-11T19:55:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:00 smithi107 ceph-mon[31712]: pgmap v50: 129 pgs: 129 active+clean; 3.0 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-11T19:55:03.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:02 smithi122 ceph-mon[36397]: pgmap v51: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-11T19:55:03.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:02 smithi145 ceph-mon[36730]: pgmap v51: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-11T19:55:03.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:02 smithi107 ceph-mon[31712]: pgmap v51: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 29 op/s 2024-03-11T19:55:04.796 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar xzf ltp-full.tgz 2024-03-11T19:55:05.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:04 smithi122 ceph-mon[36397]: pgmap v52: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-03-11T19:55:05.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:04 smithi145 ceph-mon[36730]: pgmap v52: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-03-11T19:55:05.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:04 smithi107 ceph-mon[31712]: pgmap v52: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-03-11T19:55:07.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:06 smithi122 ceph-mon[36397]: pgmap v53: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-11T19:55:07.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:06 smithi145 ceph-mon[36730]: pgmap v53: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-11T19:55:07.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:06 smithi107 ceph-mon[31712]: pgmap v53: 129 pgs: 129 active+clean; 6.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-11T19:55:08.838 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:08 smithi145 ceph-mon[36730]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 371 KiB/s wr, 27 op/s 2024-03-11T19:55:09.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:08 smithi122 ceph-mon[36397]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 371 KiB/s wr, 27 op/s 2024-03-11T19:55:09.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:08 smithi107 ceph-mon[31712]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 371 KiB/s wr, 27 op/s 2024-03-11T19:55:10.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:09 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:10.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:09 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:10.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:09 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:11.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:10 smithi122 ceph-mon[36397]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 370 KiB/s wr, 23 op/s 2024-03-11T19:55:11.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:10 smithi145 ceph-mon[36730]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 370 KiB/s wr, 23 op/s 2024-03-11T19:55:11.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:10 smithi107 ceph-mon[31712]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 370 KiB/s wr, 23 op/s 2024-03-11T19:55:13.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:12 smithi122 ceph-mon[36397]: pgmap v56: 129 pgs: 129 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-03-11T19:55:13.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:12 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:13.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:12 smithi145 ceph-mon[36730]: pgmap v56: 129 pgs: 129 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-03-11T19:55:13.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:12 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:13.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:12 smithi107 ceph-mon[31712]: pgmap v56: 129 pgs: 129 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-03-11T19:55:13.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:12 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.764+0000 7f1f47e00640 1 -- 172.21.15.107:0/1972707368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481033e0 msgr2=0x7f1f481037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.764+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1972707368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481033e0 0x7f1f481037c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009930 tx=0x7f1f3802bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 -- 172.21.15.107:0/1972707368 shutdown_connections 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1972707368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f48110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1972707368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f48103d90 0x7f1f48107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1972707368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481033e0 0x7f1f481037c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 -- 172.21.15.107:0/1972707368 >> 172.21.15.107:0/1972707368 conn(0x7f1f480fec60 msgr2=0x7f1f48101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 -- 172.21.15.107:0/1972707368 shutdown_connections 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.765+0000 7f1f47e00640 1 -- 172.21.15.107:0/1972707368 wait complete. 2024-03-11T19:55:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.766+0000 7f1f47e00640 1 Processor -- start 2024-03-11T19:55:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.766+0000 7f1f47e00640 1 -- start start 2024-03-11T19:55:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.766+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.766+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f4819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481996c0 0x7f1f48199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f47e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819a190 con 0x7f1f481996c0 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f47e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819a300 con 0x7f1f481084b0 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f47e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819a4c0 con 0x7f1f481033e0 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:42090/0 (socket says 172.21.15.107:42090) 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 -- 172.21.15.107:0/1896110673 learned_addr learned my addr 172.21.15.107:0/1896110673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f47400640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481996c0 0x7f1f48199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46000640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f4819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 -- 172.21.15.107:0/1896110673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 msgr2=0x7f1f4819fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f4819fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 -- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481996c0 msgr2=0x7f1f48199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481996c0 0x7f1f48199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.767+0000 7f1f46a00640 1 -- 172.21.15.107:0/1896110673 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40009840 con 0x7f1f481033e0 2024-03-11T19:55:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f46000640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f4819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f3802cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f3f400640 1 -- 172.21.15.107:0/1896110673 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f380042d0 con 0x7f1f481033e0 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f3f400640 1 -- 172.21.15.107:0/1896110673 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f38004430 con 0x7f1f481033e0 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38009560 con 0x7f1f481033e0 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f3f400640 1 -- 172.21.15.107:0/1896110673 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38003bf0 con 0x7f1f481033e0 2024-03-11T19:55:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.768+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f481a6750 con 0x7f1f481033e0 2024-03-11T19:55:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.772+0000 7f1f3f400640 1 -- 172.21.15.107:0/1896110673 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1f38003d50 con 0x7f1f481033e0 2024-03-11T19:55:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 Processor -- start 2024-03-11T19:55:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 -- start start 2024-03-11T19:55:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519a1620 con 0x55fa519e5ff0 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519538c0 con 0x55fa519ef570 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.773+0000 7f1f5062e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa51950df0 con 0x55fa519e6d60 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f3f400640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 0x7f1f2c0a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:55350/0 (socket says 172.21.15.107:55350) 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46000640 1 -- 172.21.15.107:0/1377521448 learned_addr learned my addr 172.21.15.107:0/1377521448 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59446/0 (socket says 172.21.15.107:59446) 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f47400640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f47400640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 0x7f1f2c0a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f3f400640 1 -- 172.21.15.107:0/1896110673 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1f380f0170 con 0x7f1f481033e0 2024-03-11T19:55:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.774+0000 7f1f46000640 1 -- 172.21.15.107:0/1377521448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 msgr2=0x55fa519e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46000640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46000640 1 -- 172.21.15.107:0/1377521448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 msgr2=0x55fa519e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46000640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46000640 1 -- 172.21.15.107:0/1377521448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa518cdb00 con 0x55fa519ef570 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f47400640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f46000640 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f340114f0 tx=0x7f1f340119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c7fc2e4dde7d23 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f47400640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 0x7f1f2c0a40a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005fd0 tx=0x7f1f40009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f33e00640 1 -- 172.21.15.107:0/1377521448 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34005020 con 0x55fa519ef570 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f33e00640 1 -- 172.21.15.107:0/1377521448 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f34005660 con 0x55fa519ef570 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.775+0000 7f1f33e00640 1 -- 172.21.15.107:0/1377521448 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400f880 con 0x55fa519ef570 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 -- 172.21.15.107:0/1377521448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 msgr2=0x55fa519f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f340114f0 tx=0x7f1f340119b0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 -- 172.21.15.107:0/1377521448 shutdown_connections 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519f1960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1377521448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 -- 172.21.15.107:0/1377521448 >> 172.21.15.107:0/1377521448 conn(0x55fa519ad4b0 msgr2=0x55fa519e0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 -- 172.21.15.107:0/1377521448 shutdown_connections 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.776+0000 7f1f5062e740 1 -- 172.21.15.107:0/1377521448 wait complete. 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.777+0000 7f1f5062e740 1 -- start start 2024-03-11T19:55:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.777+0000 7f1f5062e740 1 Processor -- start 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5194c5d0 con 0x55fa519e5ff0 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa518f1b50 con 0x55fa519ef570 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa518f17f0 con 0x55fa519e6d60 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f5062e740 10 client.0 _ll_register_callbacks cb 0x55fa51905550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f47400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f47400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:55354/0 (socket says 172.21.15.107:55354) 2024-03-11T19:55:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f47400640 1 -- 172.21.15.107:0/1650517502 learned_addr learned my addr 172.21.15.107:0/1650517502 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.778+0000 7f1f46000640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f47400640 1 -- 172.21.15.107:0/1650517502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 msgr2=0x55fa519ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f47400640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f47400640 1 -- 172.21.15.107:0/1650517502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 msgr2=0x55fa519ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f47400640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f47400640 1 -- 172.21.15.107:0/1650517502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f34009c80 con 0x55fa519ef570 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.779+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f47400640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f40011fb0 tx=0x7f1f40004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d020d426f610ae40 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4001f460 con 0x55fa519ef570 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa5191ecd0 con 0x55fa519ef570 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1f4001f5c0 con 0x55fa519ef570 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f4001f780 con 0x55fa519ef570 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f5062e740 10 client.25658 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.780+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa5194ac50 con 0x55fa519ef570 2024-03-11T19:55:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fa5191ecd0 con 0x55fa519ef570 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1f4002a9e0 con 0x55fa519ef570 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 10 client.25658 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 20 client.25658 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa519ff650 0x55fa51a01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 4 client.25658 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.781+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa51942ce0 con 0x55fa519ff650 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.782+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa519ff650 0x55fa51a01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.782+0000 7f1f46a00640 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa519ff650 0x55fa51a01a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=683926d9df687c2b server_cookie=65cdbd04b533a178 in_seq=0 out_seq=0 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.782+0000 7f1f30c00640 10 client.25658 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f30c00640 1 -- 172.21.15.107:0/1650517502 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa51942ce0 con 0x55fa519ff650 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f30c00640 10 client.25658 handle_command_reply: tid=0 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 client.25658 shutdown 2024-03-11T19:55:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 msgr2=0x55fa519ed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f40011fb0 tx=0x7f1f40004a40 comp rx=0 tx=0).stop 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 shutdown_connections 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa519ff650 0x55fa51a01a70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519ef570 0x55fa519ed030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519e6d60 0x55fa519ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 --2- 172.21.15.107:0/1650517502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 >> 172.21.15.107:0/1650517502 conn(0x55fa519ad4b0 msgr2=0x55fa519e2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f30c00640 0 client.25658 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.783+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 shutdown_connections 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.784+0000 7f1f5062e740 1 -- 172.21.15.107:0/1650517502 wait complete. 2024-03-11T19:55:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.784+0000 7f1f5062e740 20 client.25658 trim_cache size 0 max 16384 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.800+0000 7f1f5062e740 1 Processor -- start 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.800+0000 7f1f5062e740 1 -- start start 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 0x55fa519ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa51694e60 con 0x55fa519e6d60 2024-03-11T19:55:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519e1ce0 con 0x55fa519e5ff0 2024-03-11T19:55:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f5062e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519db220 con 0x55fa519ef570 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f46000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 0x55fa519ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f46000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 0x55fa519ed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59456/0 (socket says 172.21.15.107:59456) 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:55368/0 (socket says 172.21.15.107:55368) 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f46000640 1 -- 172.21.15.107:0/663093331 learned_addr learned my addr 172.21.15.107:0/663093331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.801+0000 7f1f47400640 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 -- 172.21.15.107:0/663093331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 msgr2=0x55fa519ecb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 -- 172.21.15.107:0/663093331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 msgr2=0x55fa519ed060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 0x55fa519ed060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 -- 172.21.15.107:0/663093331 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa518e7530 con 0x55fa519ef570 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f46a00640 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f47400640 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519eaea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f40010fd0 tx=0x7f1f40005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ed8e0c7c5dc902 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f33400640 1 -- 172.21.15.107:0/663093331 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40011bd0 con 0x55fa519ef570 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f33400640 1 -- 172.21.15.107:0/663093331 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f40010800 con 0x55fa519ef570 2024-03-11T19:55:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.802+0000 7f1f33400640 1 -- 172.21.15.107:0/663093331 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40004030 con 0x55fa519ef570 2024-03-11T19:55:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 -- 172.21.15.107:0/663093331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 msgr2=0x55fa519eaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519eaea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f40010fd0 tx=0x7f1f40005ac0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 -- 172.21.15.107:0/663093331 shutdown_connections 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519eaea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e6d60 0x55fa519ed060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 --2- 172.21.15.107:0/663093331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e5ff0 0x55fa519ecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 -- 172.21.15.107:0/663093331 >> 172.21.15.107:0/663093331 conn(0x55fa519ad4b0 msgr2=0x55fa519e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 -- 172.21.15.107:0/663093331 shutdown_connections 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.803+0000 7f1f5062e740 1 -- 172.21.15.107:0/663093331 wait complete. 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.804+0000 7f1f5062e740 1 -- start start 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.804+0000 7f1f5062e740 1 Processor -- start 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519f4ac0 con 0x55fa519e5ff0 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519f4c30 con 0x55fa519e6d60 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa519f4df0 con 0x55fa519ef570 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f5062e740 10 client.0 _ll_register_callbacks cb 0x7f1f40011a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:55382/0 (socket says 172.21.15.107:55382) 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 -- 172.21.15.107:0/413956740 learned_addr learned my addr 172.21.15.107:0/413956740 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f47400640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 -- 172.21.15.107:0/413956740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 msgr2=0x55fa519f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 -- 172.21.15.107:0/413956740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 msgr2=0x55fa519f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.805+0000 7f1f46000640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f46000640 1 -- 172.21.15.107:0/413956740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f481a6750 con 0x55fa519e6d60 2024-03-11T19:55:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f46a00640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f46000640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011eb0 tx=0x7f1f34005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879a12b78a1fa62e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:55:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34003900 con 0x55fa519e6d60 2024-03-11T19:55:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1f34003a60 con 0x55fa519e6d60 2024-03-11T19:55:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.806+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa51971d30 con 0x55fa519e6d60 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f34003c20 con 0x55fa519e6d60 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 10 client.25664 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa5194ac50 con 0x55fa519e6d60 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fa51971d30 con 0x55fa519e6d60 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1f34019410 con 0x55fa519e6d60 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 10 client.25664 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 20 client.25664 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa51a56c80 0x55fa51a590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 4 client.25664 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.807+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fa5164c600 con 0x55fa51a56c80 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.808+0000 7f1f46a00640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa51a56c80 0x55fa51a590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.808+0000 7f1f30c00640 10 client.25664 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:13.808+0000 7f1f46a00640 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa51a56c80 0x55fa51a590a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231904cff2b6129 server_cookie=52dd5af7398da32 in_seq=0 out_seq=0 2024-03-11T19:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.577+0000 7f1f30c00640 1 -- 172.21.15.107:0/413956740 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fa5164c600 con 0x55fa51a56c80 2024-03-11T19:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.577+0000 7f1f30c00640 10 client.25664 handle_command_reply: tid=0 2024-03-11T19:55:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.577+0000 7f1f5062e740 1 client.25664 shutdown 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 msgr2=0x55fa519f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011eb0 tx=0x7f1f34005d50 comp rx=0 tx=0).stop 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 shutdown_connections 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55fa51a56c80 0x55fa51a590a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa519e6d60 0x55fa519f6510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f30c00640 0 client.25664 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fa519ef570 0x55fa519f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 --2- 172.21.15.107:0/413956740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa519e5ff0 0x55fa519f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 >> 172.21.15.107:0/413956740 conn(0x55fa519ad4b0 msgr2=0x55fa519eae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 shutdown_connections 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 1 -- 172.21.15.107:0/413956740 wait complete. 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.578+0000 7f1f5062e740 20 client.25664 trim_cache size 0 max 16384 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:55:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 msgr2=0x7f1f2c0a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 0x7f1f2c0a40a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005fd0 tx=0x7f1f40009500 comp rx=0 tx=0).stop 2024-03-11T19:55:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 msgr2=0x7f1f4819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f3802cd60 comp rx=0 tx=0).stop 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 shutdown_connections 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1f2c0a1be0 0x7f1f2c0a40a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f481084b0 0x7f1f4819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481996c0 0x7f1f48199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 --2- 172.21.15.107:0/1896110673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f481033e0 0x7f1f4819f600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.580+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 >> 172.21.15.107:0/1896110673 conn(0x7f1f480fec60 msgr2=0x7f1f48100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.581+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 shutdown_connections 2024-03-11T19:55:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:14.581+0000 7f1f47e00640 1 -- 172.21.15.107:0/1896110673 wait complete. 2024-03-11T19:55:15.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:14 smithi122 ceph-mon[36397]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-03-11T19:55:15.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:14 smithi145 ceph-mon[36730]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-03-11T19:55:15.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:14 smithi107 ceph-mon[31712]: pgmap v57: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-03-11T19:55:17.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:16 smithi122 ceph-mon[36397]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-03-11T19:55:17.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:16 smithi145 ceph-mon[36730]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-03-11T19:55:17.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:16 smithi107 ceph-mon[31712]: pgmap v58: 129 pgs: 129 active+clean; 37 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-03-11T19:55:19.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:18 smithi122 ceph-mon[36397]: pgmap v59: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-03-11T19:55:19.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:18 smithi145 ceph-mon[36730]: pgmap v59: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-03-11T19:55:19.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:18 smithi107 ceph-mon[31712]: pgmap v59: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-03-11T19:55:21.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:20 smithi122 ceph-mon[36397]: pgmap v60: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-03-11T19:55:21.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:20 smithi145 ceph-mon[36730]: pgmap v60: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-03-11T19:55:21.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:20 smithi107 ceph-mon[31712]: pgmap v60: 129 pgs: 129 active+clean; 52 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-03-11T19:55:23.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:22 smithi122 ceph-mon[36397]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-11T19:55:23.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:22 smithi145 ceph-mon[36730]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-11T19:55:23.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:22 smithi107 ceph-mon[31712]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 391 op/s 2024-03-11T19:55:25.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:24 smithi122 ceph-mon[36397]: pgmap v62: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-11T19:55:25.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:24 smithi145 ceph-mon[36730]: pgmap v62: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-11T19:55:25.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:24 smithi107 ceph-mon[31712]: pgmap v62: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-11T19:55:27.012 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:26 smithi122 ceph-mon[36397]: pgmap v63: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-03-11T19:55:27.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:26 smithi145 ceph-mon[36730]: pgmap v63: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-03-11T19:55:27.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:26 smithi107 ceph-mon[31712]: pgmap v63: 129 pgs: 129 active+clean; 57 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 148 op/s 2024-03-11T19:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3528978241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 msgr2=0x7f8cb0110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3528978241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb0110260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009ca0 tx=0x7f8ca802c100 comp rx=0 tx=0).stop 2024-03-11T19:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3528978241 shutdown_connections 2024-03-11T19:55:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3528978241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb0103f70 0x7f8cb0107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3528978241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb0110260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3528978241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb01035c0 0x7f8cb01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3528978241 >> 172.21.15.107:0/3528978241 conn(0x7f8cb00fee40 msgr2=0x7f8cb0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3528978241 shutdown_connections 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.748+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3528978241 wait complete. 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.749+0000 7f8cb6a00640 1 Processor -- start 2024-03-11T19:55:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 -- start start 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb01035c0 0x7f8cb019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb0199740 0x7f8cb0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019a210 con 0x7f8cb0199740 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019a380 con 0x7f8cb01035c0 2024-03-11T19:55:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8cb6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019a540 con 0x7f8cb0108690 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.750+0000 7f8caf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb01035c0 0x7f8cb019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8cb4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb0199740 0x7f8cb0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59784/0 (socket says 172.21.15.107:59784) 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 -- 172.21.15.107:0/3188555200 learned_addr learned my addr 172.21.15.107:0/3188555200 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 -- 172.21.15.107:0/3188555200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb01035c0 msgr2=0x7f8cb019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb01035c0 0x7f8cb019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 -- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb0199740 msgr2=0x7f8cb0199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb0199740 0x7f8cb0199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 -- 172.21.15.107:0/3188555200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009900 con 0x7f8cb0108690 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8caf400640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00e9f0 tx=0x7f8c9c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.751+0000 7f8cacc00640 1 -- 172.21.15.107:0/3188555200 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0096c0 con 0x7f8cb0108690 2024-03-11T19:55:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.752+0000 7f8cacc00640 1 -- 172.21.15.107:0/3188555200 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c9c005500 con 0x7f8cb0108690 2024-03-11T19:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.752+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb01a64e0 con 0x7f8cb0108690 2024-03-11T19:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.752+0000 7f8cacc00640 1 -- 172.21.15.107:0/3188555200 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c010430 con 0x7f8cb0108690 2024-03-11T19:55:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.752+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb01a69c0 con 0x7f8cb0108690 2024-03-11T19:55:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.756+0000 7f8cacc00640 1 -- 172.21.15.107:0/3188555200 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f8c9c005020 con 0x7f8cb0108690 2024-03-11T19:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.756+0000 7f8cb95fd740 1 Processor -- start 2024-03-11T19:55:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.756+0000 7f8cb95fd740 1 -- start start 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 0x557cac4c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 0x557cac4cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac47f620 con 0x557cac4c4b20 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4318c0 con 0x557cac4c3db0 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac42edf0 con 0x557cac4cd330 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8caf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 0x557cac4cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 0x557cac4c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41566/0 (socket says 172.21.15.107:41566) 2024-03-11T19:55:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cb4a00640 1 -- 172.21.15.107:0/460387221 learned_addr learned my addr 172.21.15.107:0/460387221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cafe00640 1 -- 172.21.15.107:0/460387221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 msgr2=0x557cac4cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.757+0000 7f8cafe00640 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 0x557cac4cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cafe00640 1 -- 172.21.15.107:0/460387221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 msgr2=0x557cac4c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cafe00640 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cafe00640 1 -- 172.21.15.107:0/460387221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cac3abb00 con 0x557cac4c3db0 2024-03-11T19:55:27.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8caf400640 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 0x557cac4cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cacc00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 0x7f8c940a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cafe00640 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 0x557cac4c4230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000abf0 tx=0x7f8ca000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7599daaa396035a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cacc00640 1 -- 172.21.15.107:0/3188555200 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8c9c014070 con 0x7f8cb0108690 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8ca5200640 1 -- 172.21.15.107:0/460387221 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0005220 con 0x557cac4c3db0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 0x7f8c940a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8ca5200640 1 -- 172.21.15.107:0/460387221 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0004c80 con 0x557cac4c3db0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.758+0000 7f8ca5200640 1 -- 172.21.15.107:0/460387221 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0018800 con 0x557cac4c3db0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 -- 172.21.15.107:0/460387221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 msgr2=0x557cac4c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 0x557cac4c4230 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000abf0 tx=0x7f8ca000f790 comp rx=0 tx=0).stop 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 0x7f8c940a4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8037040 tx=0x7f8ca8005c50 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 -- 172.21.15.107:0/460387221 shutdown_connections 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c3db0 0x557cac4c4230 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4cd330 0x557cac4cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/460387221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 -- 172.21.15.107:0/460387221 >> 172.21.15.107:0/460387221 conn(0x557cac48b4b0 msgr2=0x557cac4beaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 -- 172.21.15.107:0/460387221 shutdown_connections 2024-03-11T19:55:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.759+0000 7f8cb95fd740 1 -- 172.21.15.107:0/460387221 wait complete. 2024-03-11T19:55:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.760+0000 7f8cb95fd740 1 -- start start 2024-03-11T19:55:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.760+0000 7f8cb95fd740 1 Processor -- start 2024-03-11T19:55:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac42a5d0 con 0x557cac4c4b20 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac3cfb50 con 0x557cac4cd330 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac3cf7f0 con 0x557cac4c3db0 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb95fd740 10 client.0 _ll_register_callbacks cb 0x557cac3e3550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41568/0 (socket says 172.21.15.107:41568) 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8caf400640 1 -- 172.21.15.107:0/1364321663 learned_addr learned my addr 172.21.15.107:0/1364321663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cafe00640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.761+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 -- 172.21.15.107:0/1364321663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 msgr2=0x557cac4ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 -- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 msgr2=0x557cac4cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 -- 172.21.15.107:0/1364321663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca000a820 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8cafe00640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8caf400640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c6a0 tx=0x7f8c9c09a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a3faa942adf4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8c99600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c09a690 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8c9c09e7e0 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cac3fccd0 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9c0a2e30 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8cb95fd740 10 client.15552 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.762+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cac428c50 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c9c0b8030 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8c9c01f060 con 0x557cac4c4b20 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 20 client.15552 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac4dd3a0 0x557cac4df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 4 client.15552 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:55:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.763+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cac420ce0 con 0x557cac4dd3a0 2024-03-11T19:55:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.764+0000 7f8cafe00640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac4dd3a0 0x557cac4df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.764+0000 7f8cafe00640 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac4dd3a0 0x557cac4df7c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1edd9fb05502d70 server_cookie=b42630f49e1a0a27 in_seq=0 out_seq=0 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.764+0000 7f8c99600640 10 client.15552 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8c99600640 1 -- 172.21.15.107:0/1364321663 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557cac420ce0 con 0x557cac4dd3a0 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8c99600640 10 client.15552 handle_command_reply: tid=0 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 client.15552 shutdown 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 msgr2=0x557cac4ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c6a0 tx=0x7f8c9c09a290 comp rx=0 tx=0).stop 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 shutdown_connections 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac4dd3a0 0x557cac4df7c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4ca890 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/1364321663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8c99600640 0 client.15552 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.765+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 >> 172.21.15.107:0/1364321663 conn(0x557cac48b4b0 msgr2=0x557cac4c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.766+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 shutdown_connections 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.766+0000 7f8cb95fd740 1 -- 172.21.15.107:0/1364321663 wait complete. 2024-03-11T19:55:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.766+0000 7f8cb95fd740 20 client.15552 trim_cache size 0 max 16384 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.793+0000 7f8cb95fd740 1 Processor -- start 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.793+0000 7f8cb95fd740 1 -- start start 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c4b20 0x557cac4cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4cd330 0x557cac4d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac172e60 con 0x557cac4cd330 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4b06a0 con 0x557cac4c4b20 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4c3080 con 0x557cac4c3db0 2024-03-11T19:55:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59800/0 (socket says 172.21.15.107:59800) 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cafe00640 1 -- 172.21.15.107:0/3134341054 learned_addr learned my addr 172.21.15.107:0/3134341054 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8caf400640 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c4b20 0x557cac4cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.794+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4cd330 0x557cac4d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 -- 172.21.15.107:0/3134341054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c4b20 msgr2=0x557cac4cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c4b20 0x557cac4cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 -- 172.21.15.107:0/3134341054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4cd330 msgr2=0x557cac4d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4cd330 0x557cac4d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 -- 172.21.15.107:0/3134341054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cac3c5530 con 0x557cac4c3db0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000ab10 tx=0x7f8ca00058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7ae49f4885cfa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.795+0000 7f8c9be00640 1 -- 172.21.15.107:0/3134341054 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0005ba0 con 0x557cac4c3db0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8c9be00640 1 -- 172.21.15.107:0/3134341054 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0005690 con 0x557cac4c3db0 2024-03-11T19:55:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8c9be00640 1 -- 172.21.15.107:0/3134341054 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0010430 con 0x557cac4c3db0 2024-03-11T19:55:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3134341054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 msgr2=0x557cac4cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000ab10 tx=0x7f8ca00058f0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3134341054 shutdown_connections 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4c4b20 0x557cac4cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4cd330 0x557cac4d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3134341054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4cca80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.796+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3134341054 >> 172.21.15.107:0/3134341054 conn(0x557cac48b4b0 msgr2=0x557cac4be5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.797+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3134341054 shutdown_connections 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.797+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3134341054 wait complete. 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.797+0000 7f8cb95fd740 1 -- start start 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.797+0000 7f8cb95fd740 1 Processor -- start 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4c75c0 con 0x557cac4c4b20 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4c7730 con 0x557cac4cd330 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cac4c78f0 con 0x557cac4c3db0 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb95fd740 10 client.0 _ll_register_callbacks cb 0x7f8c9c0969a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59812/0 (socket says 172.21.15.107:59812) 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cafe00640 1 -- 172.21.15.107:0/3688853024 learned_addr learned my addr 172.21.15.107:0/3688853024 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.798+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 -- 172.21.15.107:0/3688853024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 msgr2=0x557cac4d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 -- 172.21.15.107:0/3688853024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 msgr2=0x557cac4c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 -- 172.21.15.107:0/3688853024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca0028e70 con 0x557cac4cd330 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb4a00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4c7080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005cc0 tx=0x7f8ca80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df24ed11e6003295 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8c99600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8052070 con 0x557cac4cd330 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cac44fd30 con 0x557cac4cd330 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.799+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8ca804c420 con 0x557cac4cd330 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ca804c5e0 con 0x557cac4cd330 2024-03-11T19:55:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 10 client.25682 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cac3eb2c0 con 0x557cac4cd330 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557cac44fd30 con 0x557cac4cd330 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8ca8048040 con 0x557cac4cd330 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 10 client.25682 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 20 client.25682 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac534e90 0x557cac5372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 4 client.25682 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:55:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.800+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557cac12a600 con 0x557cac534e90 2024-03-11T19:55:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.801+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac534e90 0x557cac5372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.801+0000 7f8c99600640 10 client.25682 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.801+0000 7f8cafe00640 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac534e90 0x557cac5372b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7e758570b510e3 server_cookie=57091f3bd3a01b38 in_seq=0 out_seq=0 2024-03-11T19:55:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8c99600640 1 -- 172.21.15.107:0/3688853024 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557cac12a600 con 0x557cac534e90 2024-03-11T19:55:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8c99600640 10 client.25682 handle_command_reply: tid=0 2024-03-11T19:55:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 client.25682 shutdown 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 msgr2=0x557cac4c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4c7080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005cc0 tx=0x7f8ca80095c0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 shutdown_connections 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x557cac534e90 0x557cac5372b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cac4cd330 0x557cac4c7080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cac4c4b20 0x557cac4c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 --2- 172.21.15.107:0/3688853024 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x557cac4c3db0 0x557cac4d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 >> 172.21.15.107:0/3688853024 conn(0x557cac48b4b0 msgr2=0x557cac4c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.821+0000 7f8c99600640 0 client.25682 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 shutdown_connections 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb95fd740 1 -- 172.21.15.107:0/3688853024 wait complete. 2024-03-11T19:55:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb95fd740 20 client.25682 trim_cache size 0 max 16384 2024-03-11T19:55:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:55:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:55:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:55:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 msgr2=0x7f8c940a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 0x7f8c940a4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8037040 tx=0x7f8ca8005c50 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 msgr2=0x7f8cb019fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.822+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00e9f0 tx=0x7f8c9c00edf0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 shutdown_connections 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f8c940a1cc0 0x7f8c940a4180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cb0199740 0x7f8cb0199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8cb0108690 0x7f8cb019fbc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 --2- 172.21.15.107:0/3188555200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cb01035c0 0x7f8cb019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 >> 172.21.15.107:0/3188555200 conn(0x7f8cb00fee40 msgr2=0x7f8cb00ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 shutdown_connections 2024-03-11T19:55:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:27.823+0000 7f8cb6a00640 1 -- 172.21.15.107:0/3188555200 wait complete. 2024-03-11T19:55:29.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:28 smithi122 ceph-mon[36397]: pgmap v64: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-11T19:55:29.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:28 smithi145 ceph-mon[36730]: pgmap v64: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-11T19:55:29.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:28 smithi107 ceph-mon[31712]: pgmap v64: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 164 op/s 2024-03-11T19:55:31.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:30 smithi122 ceph-mon[36397]: pgmap v65: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-03-11T19:55:31.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:30 smithi145 ceph-mon[36730]: pgmap v65: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-03-11T19:55:31.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:30 smithi107 ceph-mon[31712]: pgmap v65: 129 pgs: 129 active+clean; 65 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-03-11T19:55:33.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:32 smithi122 ceph-mon[36397]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-03-11T19:55:33.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:32 smithi145 ceph-mon[36730]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-03-11T19:55:33.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:32 smithi107 ceph-mon[31712]: pgmap v66: 129 pgs: 129 active+clean; 81 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-03-11T19:55:35.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:34 smithi122 ceph-mon[36397]: pgmap v67: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-03-11T19:55:35.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:34 smithi145 ceph-mon[36730]: pgmap v67: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-03-11T19:55:35.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:34 smithi107 ceph-mon[31712]: pgmap v67: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2024-03-11T19:55:36.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:35 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:55:36.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:35 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:55:36.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:35 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:55:37.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:36 smithi122 ceph-mon[36397]: pgmap v68: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-03-11T19:55:37.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:36 smithi145 ceph-mon[36730]: pgmap v68: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-03-11T19:55:37.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:36 smithi107 ceph-mon[31712]: pgmap v68: 129 pgs: 129 active+clean; 82 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 119 op/s 2024-03-11T19:55:38.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:37 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:55:38.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:37 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:55:38.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:37 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:55:38.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:37 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:55:38.037 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:37 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:55:38.038 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:37 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:55:38.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:37 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:55:38.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:37 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:55:38.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:37 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:55:38.937 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:38 smithi145 ceph-mon[36730]: pgmap v69: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-11T19:55:39.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:38 smithi122 ceph-mon[36397]: pgmap v69: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-11T19:55:39.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:38 smithi107 ceph-mon[31712]: pgmap v69: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 -- 172.21.15.107:0/3391136707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 msgr2=0x7f3bd8114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 --2- 172.21.15.107:0/3391136707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd8114530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009ca0 tx=0x7f3bd002c190 comp rx=0 tx=0).stop 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 -- 172.21.15.107:0/3391136707 shutdown_connections 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 --2- 172.21.15.107:0/3391136707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073e40 0x7f3bd8111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 --2- 172.21.15.107:0/3391136707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd8114530 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 --2- 172.21.15.107:0/3391136707 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073520 0x7f3bd8073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 -- 172.21.15.107:0/3391136707 >> 172.21.15.107:0/3391136707 conn(0x7f3bd80fcc40 msgr2=0x7f3bd80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 -- 172.21.15.107:0/3391136707 shutdown_connections 2024-03-11T19:55:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.963+0000 7f3be0400640 1 -- 172.21.15.107:0/3391136707 wait complete. 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 Processor -- start 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 -- start start 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073e40 0x7f3bd819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd81a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81a2c20 con 0x7f3bd8112140 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81a2d90 con 0x7f3bd8073520 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.964+0000 7f3be0400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81a2f50 con 0x7f3bd8073e40 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:47684/0 (socket says 172.21.15.107:47684) 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 -- 172.21.15.107:0/3792583171 learned_addr learned my addr 172.21.15.107:0/3792583171 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bde400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd81a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdd000640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073e40 0x7f3bd819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 -- 172.21.15.107:0/3792583171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073e40 msgr2=0x7f3bd819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073e40 0x7f3bd819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 -- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 msgr2=0x7f3bd81a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd81a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 -- 172.21.15.107:0/3792583171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0009900 con 0x7f3bd8073520 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.965+0000 7f3bdda00640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00e9f0 tx=0x7f3bcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.966+0000 7f3bd6000640 1 -- 172.21.15.107:0/3792583171 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc009700 con 0x7f3bd8073520 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.966+0000 7f3bd6000640 1 -- 172.21.15.107:0/3792583171 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bcc00cd80 con 0x7f3bd8073520 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.966+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd81a3230 con 0x7f3bd8073520 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.966+0000 7f3bd6000640 1 -- 172.21.15.107:0/3792583171 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc018920 con 0x7f3bd8073520 2024-03-11T19:55:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.966+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd81aaab0 con 0x7f3bd8073520 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3bd6000640 1 -- 172.21.15.107:0/3792583171 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f3bcc018a80 con 0x7f3bd8073520 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 Processor -- start 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 -- start start 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d364230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d31f620 con 0x56451d364b20 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d2d18c0 con 0x56451d363db0 2024-03-11T19:55:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.969+0000 7f3be31aa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d2cedf0 con 0x56451d36d330 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55134/0 (socket says 172.21.15.107:55134) 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 -- 172.21.15.107:0/2457006005 learned_addr learned my addr 172.21.15.107:0/2457006005 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bdd000640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bdda00640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d364230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 -- 172.21.15.107:0/2457006005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 msgr2=0x56451d36f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 -- 172.21.15.107:0/2457006005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 msgr2=0x56451d364230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d364230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 -- 172.21.15.107:0/2457006005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56451d24bb00 con 0x56451d364b20 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bdd000640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bd6000640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 0x7f3bc00a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.970+0000 7f3bde400640 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0004af0 tx=0x7f3bd0005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7114f18a67e25bc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bd6000640 1 -- 172.21.15.107:0/3792583171 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3bcc014070 con 0x7f3bd8073520 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bcaa00640 1 -- 172.21.15.107:0/2457006005 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0034440 con 0x56451d364b20 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bcaa00640 1 -- 172.21.15.107:0/2457006005 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bd0003970 con 0x56451d364b20 2024-03-11T19:55:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bcaa00640 1 -- 172.21.15.107:0/2457006005 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0032d40 con 0x56451d364b20 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bdda00640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 0x7f3bc00a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3be31aa740 1 -- 172.21.15.107:0/2457006005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 msgr2=0x56451d363870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3bdda00640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 0x7f3bc00a4100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56451d365280 tx=0x7f3bcc0d1830 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.971+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0004af0 tx=0x7f3bd0005f80 comp rx=0 tx=0).stop 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 -- 172.21.15.107:0/2457006005 shutdown_connections 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d364230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2457006005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d363870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 -- 172.21.15.107:0/2457006005 >> 172.21.15.107:0/2457006005 conn(0x56451d32b4b0 msgr2=0x56451d35eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 -- 172.21.15.107:0/2457006005 shutdown_connections 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 -- 172.21.15.107:0/2457006005 wait complete. 2024-03-11T19:55:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 -- start start 2024-03-11T19:55:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.972+0000 7f3be31aa740 1 Processor -- start 2024-03-11T19:55:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d36a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d2ca5d0 con 0x56451d363db0 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d26fb50 con 0x56451d364b20 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d26f7f0 con 0x56451d36d330 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3be31aa740 10 client.0 _ll_register_callbacks cb 0x56451d283550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55142/0 (socket says 172.21.15.107:55142) 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3bde400640 1 -- 172.21.15.107:0/561111844 learned_addr learned my addr 172.21.15.107:0/561111844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.973+0000 7f3bdd000640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d36a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 -- 172.21.15.107:0/561111844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 msgr2=0x56451d36ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 -- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 msgr2=0x56451d36a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d36a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 -- 172.21.15.107:0/561111844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd8114e60 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bde400640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005e30 tx=0x7f3bd0032a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5831847e551e6921 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0032580 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3bd0003970 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.974+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56451d29ccd0 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd0003b30 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 10 client.15570 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56451d2c8c50 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56451d29ccd0 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3bd003f480 con 0x56451d363db0 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 10 client.15570 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 20 client.15570 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d37cfb0 0x56451d37f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 4 client.15570 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.975+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56451d2c0ce0 con 0x56451d37cfb0 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.976+0000 7f3bdd000640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d37cfb0 0x56451d37f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.976+0000 7f3bdd000640 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d37cfb0 0x56451d37f3d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2613cebba4845f0e server_cookie=3ecedf8e04330cc5 in_seq=0 out_seq=0 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.976+0000 7f3bbf400640 10 client.15570 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3bbf400640 1 -- 172.21.15.107:0/561111844 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56451d2c0ce0 con 0x56451d37cfb0 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3bbf400640 10 client.15570 handle_command_reply: tid=0 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 client.15570 shutdown 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 msgr2=0x56451d36a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005e30 tx=0x7f3bd0032a20 comp rx=0 tx=0).stop 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 shutdown_connections 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d37cfb0 0x56451d37f3d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d36a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d36ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3bbf400640 0 client.15570 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 --2- 172.21.15.107:0/561111844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36a360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 >> 172.21.15.107:0/561111844 conn(0x56451d32b4b0 msgr2=0x56451d3607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 shutdown_connections 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 1 -- 172.21.15.107:0/561111844 wait complete. 2024-03-11T19:55:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.977+0000 7f3be31aa740 20 client.15570 trim_cache size 0 max 16384 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.993+0000 7f3be31aa740 1 Processor -- start 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 -- start start 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d367600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d367b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d012e60 con 0x56451d363db0 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d35faa0 con 0x56451d364b20 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3be31aa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d363c20 con 0x56451d36d330 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55158/0 (socket says 172.21.15.107:55158) 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 -- 172.21.15.107:0/2446922959 learned_addr learned my addr 172.21.15.107:0/2446922959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bdd000640 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d367600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d367b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 -- 172.21.15.107:0/2446922959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 msgr2=0x56451d367b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d367b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 -- 172.21.15.107:0/2446922959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 msgr2=0x56451d367600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d367600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.994+0000 7f3bde400640 1 -- 172.21.15.107:0/2446922959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56451d265530 con 0x56451d363db0 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.995+0000 7f3bde400640 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56451d36b4f0 tx=0x7f3bd0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92b98b395c16da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.995+0000 7f3bca000640 1 -- 172.21.15.107:0/2446922959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd003fa00 con 0x56451d363db0 2024-03-11T19:55:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.995+0000 7f3bca000640 1 -- 172.21.15.107:0/2446922959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bd004d070 con 0x56451d363db0 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.995+0000 7f3bca000640 1 -- 172.21.15.107:0/2446922959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004040 con 0x56451d363db0 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- 172.21.15.107:0/2446922959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 msgr2=0x56451d36ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56451d36b4f0 tx=0x7f3bd0032f90 comp rx=0 tx=0).stop 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- 172.21.15.107:0/2446922959 shutdown_connections 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d364b20 0x56451d367600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d367b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 --2- 172.21.15.107:0/2446922959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d363db0 0x56451d36ca90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- 172.21.15.107:0/2446922959 >> 172.21.15.107:0/2446922959 conn(0x56451d32b4b0 msgr2=0x56451d368c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- 172.21.15.107:0/2446922959 shutdown_connections 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- 172.21.15.107:0/2446922959 wait complete. 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 -- start start 2024-03-11T19:55:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.996+0000 7f3be31aa740 1 Processor -- start 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d374460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d372d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d373440 con 0x56451d364b20 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d3735b0 con 0x56451d363db0 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451d3d3480 con 0x56451d36d330 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3be31aa740 10 client.0 _ll_register_callbacks cb 0x7f3bcc0d1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3bde400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d374460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55174/0 (socket says 172.21.15.107:55174) 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3bdd000640 1 -- 172.21.15.107:0/3194466218 learned_addr learned my addr 172.21.15.107:0/3194466218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.997+0000 7f3bde400640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d372d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 -- 172.21.15.107:0/3194466218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 msgr2=0x56451d372d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d372d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 -- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 msgr2=0x56451d374460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d374460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 -- 172.21.15.107:0/3194466218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56451d2c8c50 con 0x56451d364b20 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bde400640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d374460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bdd000640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4004700 tx=0x7f3bc401f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7ce72f7a69ec38 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4017030 con 0x56451d364b20 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3bc4010960 con 0x56451d364b20 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56451d2efd30 con 0x56451d364b20 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bc4010b20 con 0x56451d364b20 2024-03-11T19:55:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.998+0000 7f3be31aa740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56451d28b2c0 con 0x56451d364b20 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56451d2efd30 con 0x56451d364b20 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3bc4004aa0 con 0x56451d364b20 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 20 client.15576 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d3d4cd0 0x56451d3d70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 4 client.15576 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56451cfca600 con 0x56451d3d4cd0 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:40.999+0000 7f3bde400640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d3d4cd0 0x56451d3d70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.000+0000 7f3bde400640 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d3d4cd0 0x56451d3d70f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfc97c398aff8ad server_cookie=5a3ddc73bdb278b2 in_seq=0 out_seq=0 2024-03-11T19:55:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.000+0000 7f3bbf400640 10 client.15576 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:41.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:40 smithi122 ceph-mon[36397]: pgmap v70: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-03-11T19:55:41.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:40 smithi145 ceph-mon[36730]: pgmap v70: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-03-11T19:55:41.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:40 smithi107 ceph-mon[31712]: pgmap v70: 129 pgs: 129 active+clean; 105 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-03-11T19:55:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3bbf400640 1 -- 172.21.15.107:0/3194466218 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56451cfca600 con 0x56451d3d4cd0 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3bbf400640 10 client.15576 handle_command_reply: tid=0 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 client.15576 shutdown 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 msgr2=0x56451d372800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4004700 tx=0x7f3bc401f890 comp rx=0 tx=0).stop 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 shutdown_connections 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x56451d3d4cd0 0x56451d3d70f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56451d36d330 0x56451d372d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56451d364b20 0x56451d372800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3bbf400640 0 client.15576 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 --2- 172.21.15.107:0/3194466218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56451d363db0 0x56451d374460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.077+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 >> 172.21.15.107:0/3194466218 conn(0x56451d32b4b0 msgr2=0x56451d35dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 shutdown_connections 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be31aa740 1 -- 172.21.15.107:0/3194466218 wait complete. 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be31aa740 20 client.15576 trim_cache size 0 max 16384 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:55:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:55:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:55:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 msgr2=0x7f3bc00a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 0x7f3bc00a4100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x56451d365280 tx=0x7f3bcc0d1830 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 msgr2=0x7f3bd819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00e9f0 tx=0x7f3bcc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 shutdown_connections 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f3bc00a1c40 0x7f3bc00a4100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bd8112140 0x7f3bd81a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd8073e40 0x7f3bd819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 --2- 172.21.15.107:0/3792583171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bd8073520 0x7f3bd819dd60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.078+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 >> 172.21.15.107:0/3792583171 conn(0x7f3bd80fcc40 msgr2=0x7f3bd80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.079+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 shutdown_connections 2024-03-11T19:55:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:41.079+0000 7f3be0400640 1 -- 172.21.15.107:0/3792583171 wait complete. 2024-03-11T19:55:43.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:42 smithi122 ceph-mon[36397]: pgmap v71: 129 pgs: 129 active+clean; 108 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-03-11T19:55:43.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:42 smithi145 ceph-mon[36730]: pgmap v71: 129 pgs: 129 active+clean; 108 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-03-11T19:55:43.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:42 smithi107 ceph-mon[31712]: pgmap v71: 129 pgs: 129 active+clean; 108 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-03-11T19:55:45.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:44 smithi122 ceph-mon[36397]: pgmap v72: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-03-11T19:55:45.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:44 smithi145 ceph-mon[36730]: pgmap v72: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-03-11T19:55:45.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:44 smithi107 ceph-mon[31712]: pgmap v72: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 403 op/s 2024-03-11T19:55:45.610 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-11T19:55:45.611 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:55:45.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-03-11T19:55:45.633 INFO:tasks.workunit.client.0.smithi107.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-11T19:55:46.471 INFO:tasks.workunit.client.0.smithi107.stderr:++ readlink -f fsstress 2024-03-11T19:55:46.474 INFO:tasks.workunit.client.0.smithi107.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-11T19:55:46.474 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-03-11T19:55:46.475 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:55:46.475 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:55:46.475 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-03-11T19:55:46.475 INFO:tasks.workunit.client.0.smithi107.stderr:++ mktemp -d -p . 2024-03-11T19:55:46.480 INFO:tasks.workunit.client.0.smithi107.stderr:+ T=./tmp.vcE6wKMJIu 2024-03-11T19:55:46.480 INFO:tasks.workunit.client.0.smithi107.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.vcE6wKMJIu -l 1 -n 1000 -p 10 -v 2024-03-11T19:55:46.484 INFO:tasks.workunit.client.0.smithi107.stdout:seed = 1709703627 2024-03-11T19:55:46.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/0: dread - no filename 2024-03-11T19:55:46.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/1: truncate - no filename 2024-03-11T19:55:46.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/2: write - no filename 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:1/0: readlink - no filename 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/3: mknod c0 0 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/4: chown c0 112982 1 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/5: rmdir - no directory 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/6: readlink - no filename 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/0: dread - no filename 2024-03-11T19:55:46.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/1: write - no filename 2024-03-11T19:55:46.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/2: write - no filename 2024-03-11T19:55:46.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/1: symlink l0 0 2024-03-11T19:55:46.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/2: truncate - no filename 2024-03-11T19:55:46.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/0: chown . 474890 1 2024-03-11T19:55:46.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/1: write - no filename 2024-03-11T19:55:46.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/2: fsync - no filename 2024-03-11T19:55:46.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/3: symlink l1 0 2024-03-11T19:55:46.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/4: truncate - no filename 2024-03-11T19:55:46.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/0: dread - no filename 2024-03-11T19:55:46.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/1: rmdir - no directory 2024-03-11T19:55:46.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/3: mknod c0 0 2024-03-11T19:55:46.501 INFO:tasks.workunit.client.0.smithi107.stdout:1/5: creat f2 x:0 0 0 2024-03-11T19:55:46.501 INFO:tasks.workunit.client.0.smithi107.stdout:1/6: dread - f2 zero size 2024-03-11T19:55:46.502 INFO:tasks.workunit.client.0.smithi107.stdout:1/7: creat f3 x:0 0 0 2024-03-11T19:55:46.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/0: chown . 56675797 1 2024-03-11T19:55:46.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/1: dread - no filename 2024-03-11T19:55:46.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/7: getdents . 0 2024-03-11T19:55:46.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/2: creat f0 x:0 0 0 2024-03-11T19:55:46.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/8: unlink f2 0 2024-03-11T19:55:46.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/9: chown l1 1657164 1 2024-03-11T19:55:46.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/4: mkdir d1 0 2024-03-11T19:55:46.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/10: write f3 [707365,11447] 0 2024-03-11T19:55:46.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/5: rename d1 to d1/d2 22 2024-03-11T19:55:46.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/2: mknod c0 0 2024-03-11T19:55:46.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/3: chown c0 2 1 2024-03-11T19:55:46.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/0: getdents . 0 2024-03-11T19:55:46.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/8: symlink l1 0 2024-03-11T19:55:46.527 INFO:tasks.workunit.client.0.smithi107.stdout:3/6: mknod d1/c3 0 2024-03-11T19:55:46.527 INFO:tasks.workunit.client.0.smithi107.stdout:3/7: dwrite - no filename 2024-03-11T19:55:46.532 INFO:tasks.workunit.client.0.smithi107.stdout:5/4: rename c0 to c1 0 2024-03-11T19:55:46.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/1: creat f0 x:0 0 0 2024-03-11T19:55:46.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/2: write f0 [89062,62434] 0 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/0: truncate - no filename 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/1: dwrite - no filename 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/2: write - no filename 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/3: read - no filename 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:3/8: unlink d1/c3 0 2024-03-11T19:55:46.535 INFO:tasks.workunit.client.0.smithi107.stdout:3/9: dwrite - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/9: symlink l2 0 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/10: dwrite - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/11: write - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/5: symlink l2 0 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/6: dread - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/7: write - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/8: chown c1 611490 1 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/9: dwrite - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/10: read - no filename 2024-03-11T19:55:46.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/11: dread - no filename 2024-03-11T19:55:46.540 INFO:tasks.workunit.client.0.smithi107.stdout:7/4: symlink l0 0 2024-03-11T19:55:46.540 INFO:tasks.workunit.client.0.smithi107.stdout:3/10: symlink d1/l4 0 2024-03-11T19:55:46.540 INFO:tasks.workunit.client.0.smithi107.stdout:3/11: fsync - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/12: fdatasync - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/13: dwrite - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/14: dwrite - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:3/15: dwrite - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/0: write - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/1: write - no filename 2024-03-11T19:55:46.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/2: readlink - no filename 2024-03-11T19:55:46.542 INFO:tasks.workunit.client.0.smithi107.stdout:0/12: chown c0 9856474 1 2024-03-11T19:55:46.542 INFO:tasks.workunit.client.0.smithi107.stdout:0/13: dread - no filename 2024-03-11T19:55:46.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/12: mknod c3 0 2024-03-11T19:55:46.546 INFO:tasks.workunit.client.0.smithi107.stdout:7/5: mknod c1 0 2024-03-11T19:55:46.546 INFO:tasks.workunit.client.0.smithi107.stdout:7/6: truncate - no filename 2024-03-11T19:55:46.546 INFO:tasks.workunit.client.0.smithi107.stdout:7/7: fdatasync - no filename 2024-03-11T19:55:46.546 INFO:tasks.workunit.client.0.smithi107.stdout:7/8: rmdir - no directory 2024-03-11T19:55:46.546 INFO:tasks.workunit.client.0.smithi107.stdout:7/9: write - no filename 2024-03-11T19:55:46.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/11: write f3 [628542,18148] 0 2024-03-11T19:55:46.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/12: read f3 [660232,14513] 0 2024-03-11T19:55:46.563 INFO:tasks.workunit.client.0.smithi107.stdout:8/3: creat f0 x:0 0 0 2024-03-11T19:55:46.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/0: readlink - no filename 2024-03-11T19:55:46.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/1: read - no filename 2024-03-11T19:55:46.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/2: dread - no filename 2024-03-11T19:55:46.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/14: creat f3 x:0 0 0 2024-03-11T19:55:46.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/15: write f3 [817115,55601] 0 2024-03-11T19:55:46.568 INFO:tasks.workunit.client.0.smithi107.stdout:5/13: creat f4 x:0 0 0 2024-03-11T19:55:46.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/14: write f4 [823717,78974] 0 2024-03-11T19:55:46.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/10: unlink c1 0 2024-03-11T19:55:46.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/11: dwrite - no filename 2024-03-11T19:55:46.583 INFO:tasks.workunit.client.0.smithi107.stdout:9/3: creat f0 x:0 0 0 2024-03-11T19:55:46.583 INFO:tasks.workunit.client.0.smithi107.stdout:9/4: chown f0 407219437 1 2024-03-11T19:55:46.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/12: unlink l0 0 2024-03-11T19:55:46.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/5: link f0 f1 0 2024-03-11T19:55:46.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/6: mknod c2 0 2024-03-11T19:55:46.595 INFO:tasks.workunit.client.0.smithi107.stdout:9/7: symlink l3 0 2024-03-11T19:55:46.630 INFO:tasks.workunit.client.0.smithi107.stdout:3/16: getdents d1 0 2024-03-11T19:55:46.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/4: getdents . 0 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/17: creat d1/f5 x:0 0 0 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/18: rename d1 to d1/d6 22 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/19: chown d1 4987676 1 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/20: fdatasync d1/f5 0 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/21: creat d1/f7 x:0 0 0 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/22: truncate d1/f5 503246 0 2024-03-11T19:55:46.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/23: fdatasync d1/f5 0 2024-03-11T19:55:46.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/24: symlink d1/l8 0 2024-03-11T19:55:46.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/3: dwrite f0 [0,4194304] 0 2024-03-11T19:55:47.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:46 smithi122 ceph-mon[36397]: pgmap v73: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 380 op/s 2024-03-11T19:55:47.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:46 smithi145 ceph-mon[36730]: pgmap v73: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 380 op/s 2024-03-11T19:55:47.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:46 smithi107 ceph-mon[31712]: pgmap v73: 129 pgs: 129 active+clean; 109 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 380 op/s 2024-03-11T19:55:47.136 INFO:tasks.workunit.client.0.smithi107.stdout:8/5: dwrite f0 [0,4194304] 0 2024-03-11T19:55:47.136 INFO:tasks.workunit.client.0.smithi107.stdout:8/6: readlink - no filename 2024-03-11T19:55:47.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/8: dwrite f1 [0,4194304] 0 2024-03-11T19:55:47.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/9: fdatasync f0 0 2024-03-11T19:55:47.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/10: readlink l3 0 2024-03-11T19:55:47.144 INFO:tasks.workunit.client.0.smithi107.stdout:9/11: mkdir d4 0 2024-03-11T19:55:47.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/12: rename c2 to d4/c5 0 2024-03-11T19:55:47.313 INFO:tasks.workunit.client.0.smithi107.stdout:3/25: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:47.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/4: write f0 [3500450,27769] 0 2024-03-11T19:55:47.355 INFO:tasks.workunit.client.0.smithi107.stdout:4/5: readlink - no filename 2024-03-11T19:55:47.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/7: dwrite f0 [0,4194304] 0 2024-03-11T19:55:47.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/26: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:47.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/13: dread f1 [0,4194304] 0 2024-03-11T19:55:47.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/6: fsync f0 0 2024-03-11T19:55:47.460 INFO:tasks.workunit.client.0.smithi107.stdout:8/8: write f0 [2304668,22873] 0 2024-03-11T19:55:47.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/13: dread f3 [0,4194304] 0 2024-03-11T19:55:47.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/14: creat f4 x:0 0 0 2024-03-11T19:55:47.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/15: dread - f4 zero size 2024-03-11T19:55:47.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/9: unlink f0 0 2024-03-11T19:55:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/10: dwrite - no filename 2024-03-11T19:55:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/16: rename f3 to f5 0 2024-03-11T19:55:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/17: stat l0 0 2024-03-11T19:55:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/18: mkdir d6 0 2024-03-11T19:55:47.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/19: fdatasync f5 0 2024-03-11T19:55:47.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/15: dread f4 [0,4194304] 0 2024-03-11T19:55:47.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/16: creat f5 x:0 0 0 2024-03-11T19:55:47.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/17: dread f4 [0,4194304] 0 2024-03-11T19:55:47.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/20: dwrite f4 [0,4194304] 0 2024-03-11T19:55:47.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/18: dread f4 [0,4194304] 0 2024-03-11T19:55:47.552 INFO:tasks.workunit.client.0.smithi107.stdout:4/7: dread f0 [0,4194304] 0 2024-03-11T19:55:47.552 INFO:tasks.workunit.client.0.smithi107.stdout:3/27: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/8: write f0 [4344027,57585] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/9: creat f1 x:0 0 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:3/28: write d1/f5 [4970827,104127] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:0/16: dwrite f3 [0,4194304] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/10: write f1 [361429,39228] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/11: write f0 [1977356,96769] 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/12: stat f0 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/13: truncate f1 899626 0 2024-03-11T19:55:47.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/14: creat f2 x:0 0 0 2024-03-11T19:55:47.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/21: mknod d6/c7 0 2024-03-11T19:55:47.557 INFO:tasks.workunit.client.0.smithi107.stdout:5/19: mknod c6 0 2024-03-11T19:55:47.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/29: link d1/l4 d1/l9 0 2024-03-11T19:55:47.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/30: rename d1/f7 to d1/fa 0 2024-03-11T19:55:47.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/31: chown d1/fa 550 1 2024-03-11T19:55:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/32: mknod d1/cb 0 2024-03-11T19:55:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/22: truncate f4 1862128 0 2024-03-11T19:55:47.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/23: mknod d6/c8 0 2024-03-11T19:55:47.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/24: unlink d6/c7 0 2024-03-11T19:55:47.606 INFO:tasks.workunit.client.0.smithi107.stdout:0/17: dwrite f3 [0,4194304] 0 2024-03-11T19:55:47.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/18: link l2 l4 0 2024-03-11T19:55:47.609 INFO:tasks.workunit.client.0.smithi107.stdout:0/19: creat f5 x:0 0 0 2024-03-11T19:55:47.613 INFO:tasks.workunit.client.0.smithi107.stdout:0/20: symlink l6 0 2024-03-11T19:55:47.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/21: symlink l7 0 2024-03-11T19:55:47.620 INFO:tasks.workunit.client.0.smithi107.stdout:0/22: rename l4 to l8 0 2024-03-11T19:55:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/15: dwrite f0 [0,4194304] 0 2024-03-11T19:55:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/16: creat f3 x:0 0 0 2024-03-11T19:55:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/17: rmdir - no directory 2024-03-11T19:55:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/18: rmdir - no directory 2024-03-11T19:55:47.627 INFO:tasks.workunit.client.0.smithi107.stdout:0/23: truncate f3 878565 0 2024-03-11T19:55:47.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/24: unlink c0 0 2024-03-11T19:55:47.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/25: chown l1 87 1 2024-03-11T19:55:47.630 INFO:tasks.workunit.client.0.smithi107.stdout:0/26: mknod c9 0 2024-03-11T19:55:47.631 INFO:tasks.workunit.client.0.smithi107.stdout:0/27: mknod ca 0 2024-03-11T19:55:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/28: symlink lb 0 2024-03-11T19:55:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/33: fdatasync d1/f5 0 2024-03-11T19:55:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/34: readlink d1/l8 0 2024-03-11T19:55:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/20: dwrite f4 [0,4194304] 0 2024-03-11T19:55:47.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/29: write f5 [721723,129022] 0 2024-03-11T19:55:47.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/35: symlink d1/lc 0 2024-03-11T19:55:47.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/36: fsync d1/fa 0 2024-03-11T19:55:47.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/21: unlink c3 0 2024-03-11T19:55:47.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/30: mknod cc 0 2024-03-11T19:55:47.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/31: stat f3 0 2024-03-11T19:55:47.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/22: rename l2 to l7 0 2024-03-11T19:55:47.643 INFO:tasks.workunit.client.0.smithi107.stdout:5/23: write f5 [630681,80271] 0 2024-03-11T19:55:47.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/32: creat fd x:0 0 0 2024-03-11T19:55:47.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/33: rmdir - no directory 2024-03-11T19:55:47.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/3: dread f0 [0,4194304] 0 2024-03-11T19:55:47.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/4: chown f0 1874378 1 2024-03-11T19:55:47.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/5: readlink - no filename 2024-03-11T19:55:47.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/6: creat f1 x:0 0 0 2024-03-11T19:55:47.658 INFO:tasks.workunit.client.0.smithi107.stdout:3/37: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:47.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/38: chown c0 8662 1 2024-03-11T19:55:47.660 INFO:tasks.workunit.client.0.smithi107.stdout:5/24: dread f5 [0,4194304] 0 2024-03-11T19:55:47.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/7: link f0 f2 0 2024-03-11T19:55:47.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/8: stat f0 0 2024-03-11T19:55:47.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/9: creat f3 x:0 0 0 2024-03-11T19:55:47.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/10: dread - f1 zero size 2024-03-11T19:55:47.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/39: getdents d1 0 2024-03-11T19:55:47.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/40: fsync d1/fa 0 2024-03-11T19:55:47.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/41: write d1/fa [687582,5514] 0 2024-03-11T19:55:47.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/25: dwrite f5 [0,4194304] 0 2024-03-11T19:55:47.868 INFO:tasks.workunit.client.0.smithi107.stdout:3/42: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:47.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/43: write d1/f5 [2120293,47733] 0 2024-03-11T19:55:47.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/44: symlink d1/ld 0 2024-03-11T19:55:47.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/45: chown d1 27731 1 2024-03-11T19:55:47.886 INFO:tasks.workunit.client.0.smithi107.stdout:5/26: dread f4 [0,4194304] 0 2024-03-11T19:55:47.887 INFO:tasks.workunit.client.0.smithi107.stdout:5/27: chown l7 154169855 1 2024-03-11T19:55:47.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/46: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:47.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/47: dwrite d1/f5 [4194304,4194304] 0 2024-03-11T19:55:47.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/48: rename d1 to d1/de 22 2024-03-11T19:55:47.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/49: mknod d1/cf 0 2024-03-11T19:55:47.949 INFO:tasks.workunit.client.0.smithi107.stdout:3/50: mkdir d1/d10 0 2024-03-11T19:55:47.997 INFO:tasks.workunit.client.0.smithi107.stdout:3/51: dwrite d1/f5 [4194304,4194304] 0 2024-03-11T19:55:48.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/13: sync 2024-03-11T19:55:48.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/14: link - no file 2024-03-11T19:55:48.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/15: chown . 56 1 2024-03-11T19:55:48.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/16: creat f2 x:0 0 0 2024-03-11T19:55:48.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/17: write f2 [580935,26989] 0 2024-03-11T19:55:48.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/18: mknod c3 0 2024-03-11T19:55:48.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/19: unlink c3 0 2024-03-11T19:55:48.581 INFO:tasks.workunit.client.0.smithi107.stdout:7/20: rename f2 to f4 0 2024-03-11T19:55:48.581 INFO:tasks.workunit.client.0.smithi107.stdout:7/21: read f4 [592179,76199] 0 2024-03-11T19:55:48.600 INFO:tasks.workunit.client.0.smithi107.stdout:7/22: dread f4 [0,4194304] 0 2024-03-11T19:55:48.606 INFO:tasks.workunit.client.0.smithi107.stdout:7/23: write f4 [115368,24797] 0 2024-03-11T19:55:48.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/24: dread f4 [0,4194304] 0 2024-03-11T19:55:48.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/25: chown f4 12638 1 2024-03-11T19:55:48.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/26: stat f4 0 2024-03-11T19:55:48.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/27: rename f4 to f5 0 2024-03-11T19:55:48.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/28: mkdir d6 0 2024-03-11T19:55:48.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/29: readlink - no filename 2024-03-11T19:55:48.716 INFO:tasks.workunit.client.0.smithi107.stdout:7/30: dwrite f5 [0,4194304] 0 2024-03-11T19:55:48.748 INFO:tasks.workunit.client.0.smithi107.stdout:7/31: write f5 [826965,43956] 0 2024-03-11T19:55:48.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/32: dwrite f5 [0,4194304] 0 2024-03-11T19:55:48.807 INFO:tasks.workunit.client.0.smithi107.stdout:7/33: mkdir d6/d7 0 2024-03-11T19:55:48.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/34: dread f5 [0,4194304] 0 2024-03-11T19:55:48.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/35: fdatasync f5 0 2024-03-11T19:55:48.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/36: write f5 [4041617,68262] 0 2024-03-11T19:55:48.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/3: sync 2024-03-11T19:55:48.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/4: rename - no filename 2024-03-11T19:55:48.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/5: truncate - no filename 2024-03-11T19:55:48.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/6: fsync - no filename 2024-03-11T19:55:48.936 INFO:tasks.workunit.client.0.smithi107.stdout:7/37: dwrite f5 [0,4194304] 0 2024-03-11T19:55:48.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/7: creat f0 x:0 0 0 2024-03-11T19:55:48.937 INFO:tasks.workunit.client.0.smithi107.stdout:7/38: creat d6/d7/f8 x:0 0 0 2024-03-11T19:55:48.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/8: mkdir d1 0 2024-03-11T19:55:48.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/9: write f0 [746658,115715] 0 2024-03-11T19:55:48.965 INFO:tasks.workunit.client.0.smithi107.stdout:2/10: dread f0 [0,4194304] 0 2024-03-11T19:55:48.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/14: sync 2024-03-11T19:55:48.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/15: rename d4 to d4/d6 22 2024-03-11T19:55:48.979 INFO:tasks.workunit.client.0.smithi107.stdout:7/39: dread f5 [0,4194304] 0 2024-03-11T19:55:48.980 INFO:tasks.workunit.client.0.smithi107.stdout:7/40: readlink - no filename 2024-03-11T19:55:49.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:48 smithi122 ceph-mon[36397]: pgmap v74: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-03-11T19:55:49.034 INFO:tasks.workunit.client.0.smithi107.stdout:7/41: read f5 [2264200,100331] 0 2024-03-11T19:55:49.034 INFO:tasks.workunit.client.0.smithi107.stdout:7/42: fsync f5 0 2024-03-11T19:55:49.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:48 smithi145 ceph-mon[36730]: pgmap v74: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-03-11T19:55:49.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:48 smithi107 ceph-mon[31712]: pgmap v74: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-03-11T19:55:49.044 INFO:tasks.workunit.client.0.smithi107.stdout:2/11: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:2/12: stat d1 0 2024-03-11T19:55:49.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/13: dread f0 [0,4194304] 0 2024-03-11T19:55:49.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/14: fsync f0 0 2024-03-11T19:55:49.080 INFO:tasks.workunit.client.0.smithi107.stdout:2/15: symlink d1/l2 0 2024-03-11T19:55:49.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/16: unlink f0 0 2024-03-11T19:55:49.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/17: write - no filename 2024-03-11T19:55:49.083 INFO:tasks.workunit.client.0.smithi107.stdout:2/18: mknod d1/c3 0 2024-03-11T19:55:49.087 INFO:tasks.workunit.client.0.smithi107.stdout:2/19: getdents d1 0 2024-03-11T19:55:49.087 INFO:tasks.workunit.client.0.smithi107.stdout:2/20: dwrite - no filename 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/11: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/34: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:5/28: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/11: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/52: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/25: sync 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/12: write - no filename 2024-03-11T19:55:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/13: dwrite - no filename 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/35: fdatasync f5 0 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/12: write f1 [702159,20650] 0 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/53: write d1/f5 [2353623,87008] 0 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/54: fdatasync d1/fa 0 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/55: chown d1/cb 2422302 1 2024-03-11T19:55:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:8/14: mknod c1 0 2024-03-11T19:55:49.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/43: dwrite f5 [0,4194304] 0 2024-03-11T19:55:49.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/56: link c0 d1/d10/c11 0 2024-03-11T19:55:49.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/15: rename c1 to c2 0 2024-03-11T19:55:49.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/16: chown c2 25776 1 2024-03-11T19:55:49.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/36: mkdir de 0 2024-03-11T19:55:49.121 INFO:tasks.workunit.client.0.smithi107.stdout:4/19: sync 2024-03-11T19:55:49.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/13: truncate f1 282097 0 2024-03-11T19:55:49.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/37: mknod de/cf 0 2024-03-11T19:55:49.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/17: creat f3 x:0 0 0 2024-03-11T19:55:49.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/18: creat f4 x:0 0 0 2024-03-11T19:55:49.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/38: write f5 [1862114,117465] 0 2024-03-11T19:55:49.124 INFO:tasks.workunit.client.0.smithi107.stdout:0/39: truncate fd 819066 0 2024-03-11T19:55:49.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/19: symlink l5 0 2024-03-11T19:55:49.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/20: fdatasync f4 0 2024-03-11T19:55:49.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/21: dread - f4 zero size 2024-03-11T19:55:49.127 INFO:tasks.workunit.client.0.smithi107.stdout:0/40: rename c9 to de/c10 0 2024-03-11T19:55:49.128 INFO:tasks.workunit.client.0.smithi107.stdout:8/22: write f4 [263428,86205] 0 2024-03-11T19:55:49.128 INFO:tasks.workunit.client.0.smithi107.stdout:8/23: truncate f3 194542 0 2024-03-11T19:55:49.128 INFO:tasks.workunit.client.0.smithi107.stdout:6/14: truncate f1 1267149 0 2024-03-11T19:55:49.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/41: mkdir de/d11 0 2024-03-11T19:55:49.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/42: chown de 53 1 2024-03-11T19:55:49.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/43: truncate fd 1101199 0 2024-03-11T19:55:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/26: dread f4 [0,4194304] 0 2024-03-11T19:55:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/27: read f5 [372846,93362] 0 2024-03-11T19:55:49.169 INFO:tasks.workunit.client.0.smithi107.stdout:7/44: dread f5 [0,4194304] 0 2024-03-11T19:55:49.171 INFO:tasks.workunit.client.0.smithi107.stdout:6/15: read f1 [141522,112655] 0 2024-03-11T19:55:49.171 INFO:tasks.workunit.client.0.smithi107.stdout:5/29: dread f4 [0,4194304] 0 2024-03-11T19:55:49.172 INFO:tasks.workunit.client.0.smithi107.stdout:4/20: dread f0 [0,4194304] 0 2024-03-11T19:55:49.172 INFO:tasks.workunit.client.0.smithi107.stdout:6/16: rename f1 to f4 0 2024-03-11T19:55:49.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/30: rename c6 to c8 0 2024-03-11T19:55:49.175 INFO:tasks.workunit.client.0.smithi107.stdout:7/45: rename d6/d7 to d6/d9 0 2024-03-11T19:55:49.175 INFO:tasks.workunit.client.0.smithi107.stdout:1/28: dread f4 [0,4194304] 0 2024-03-11T19:55:49.176 INFO:tasks.workunit.client.0.smithi107.stdout:7/46: mkdir d6/da 0 2024-03-11T19:55:49.180 INFO:tasks.workunit.client.0.smithi107.stdout:8/24: dread f4 [0,4194304] 0 2024-03-11T19:55:49.184 INFO:tasks.workunit.client.0.smithi107.stdout:6/17: dread f0 [0,4194304] 0 2024-03-11T19:55:49.184 INFO:tasks.workunit.client.0.smithi107.stdout:6/18: write f4 [1359893,89950] 0 2024-03-11T19:55:49.188 INFO:tasks.workunit.client.0.smithi107.stdout:9/16: sync 2024-03-11T19:55:49.188 INFO:tasks.workunit.client.0.smithi107.stdout:9/17: stat d4 0 2024-03-11T19:55:49.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/25: dread f3 [0,4194304] 0 2024-03-11T19:55:49.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/26: symlink l6 0 2024-03-11T19:55:49.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/27: write f4 [706612,36691] 0 2024-03-11T19:55:49.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/28: rename l6 to l7 0 2024-03-11T19:55:49.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/29: dread f4 [0,4194304] 0 2024-03-11T19:55:49.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/30: truncate f4 2488045 0 2024-03-11T19:55:49.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/29: mknod c8 0 2024-03-11T19:55:49.196 INFO:tasks.workunit.client.0.smithi107.stdout:0/44: dwrite fd [0,4194304] 0 2024-03-11T19:55:49.196 INFO:tasks.workunit.client.0.smithi107.stdout:3/57: dread d1/fa [0,4194304] 0 2024-03-11T19:55:49.196 INFO:tasks.workunit.client.0.smithi107.stdout:8/30: write f3 [612800,119305] 0 2024-03-11T19:55:49.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/19: truncate f4 848463 0 2024-03-11T19:55:49.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/20: fdatasync f0 0 2024-03-11T19:55:49.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/21: readlink - no filename 2024-03-11T19:55:49.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/31: symlink l9 0 2024-03-11T19:55:49.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/32: rmdir - no directory 2024-03-11T19:55:49.199 INFO:tasks.workunit.client.0.smithi107.stdout:4/21: dread f0 [0,4194304] 0 2024-03-11T19:55:49.202 INFO:tasks.workunit.client.0.smithi107.stdout:6/22: rename f4 to f5 0 2024-03-11T19:55:49.202 INFO:tasks.workunit.client.0.smithi107.stdout:6/23: stat f0 0 2024-03-11T19:55:49.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/24: getdents . 0 2024-03-11T19:55:49.211 INFO:tasks.workunit.client.0.smithi107.stdout:5/31: write f4 [180537,82775] 0 2024-03-11T19:55:49.214 INFO:tasks.workunit.client.0.smithi107.stdout:4/22: dread f1 [0,4194304] 0 2024-03-11T19:55:49.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/23: mkdir d4 0 2024-03-11T19:55:49.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/24: symlink d4/l5 0 2024-03-11T19:55:49.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/25: creat d4/f6 x:0 0 0 2024-03-11T19:55:49.218 INFO:tasks.workunit.client.0.smithi107.stdout:4/26: write f3 [640499,24146] 0 2024-03-11T19:55:49.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/27: mkdir d4/d7 0 2024-03-11T19:55:49.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/28: rename f1 to d4/d7/f8 0 2024-03-11T19:55:49.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/29: symlink d4/d7/l9 0 2024-03-11T19:55:49.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/30: mknod d4/d7/ca 0 2024-03-11T19:55:49.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/31: chown d4/f6 382 1 2024-03-11T19:55:49.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/18: dwrite f1 [0,4194304] 0 2024-03-11T19:55:49.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/19: stat f0 0 2024-03-11T19:55:49.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/20: chown d4 11884 1 2024-03-11T19:55:49.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/58: dread d1/fa [0,4194304] 0 2024-03-11T19:55:49.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/59: mkdir d1/d12 0 2024-03-11T19:55:49.250 INFO:tasks.workunit.client.0.smithi107.stdout:0/45: dwrite fd [0,4194304] 0 2024-03-11T19:55:49.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/46: symlink de/l12 0 2024-03-11T19:55:49.261 INFO:tasks.workunit.client.0.smithi107.stdout:1/31: dwrite f5 [0,4194304] 0 2024-03-11T19:55:49.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/32: creat d6/f9 x:0 0 0 2024-03-11T19:55:49.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/21: dread f0 [0,4194304] 0 2024-03-11T19:55:49.270 INFO:tasks.workunit.client.0.smithi107.stdout:9/22: creat d4/f7 x:0 0 0 2024-03-11T19:55:49.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/23: mknod d4/c8 0 2024-03-11T19:55:49.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/24: stat d4 0 2024-03-11T19:55:49.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/25: stat d4/c8 0 2024-03-11T19:55:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/25: dwrite f2 [0,4194304] 0 2024-03-11T19:55:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/26: getdents . 0 2024-03-11T19:55:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/27: chown f3 96366 1 2024-03-11T19:55:49.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/28: getdents . 0 2024-03-11T19:55:49.278 INFO:tasks.workunit.client.0.smithi107.stdout:6/29: truncate f5 267364 0 2024-03-11T19:55:49.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/30: creat f6 x:0 0 0 2024-03-11T19:55:49.283 INFO:tasks.workunit.client.0.smithi107.stdout:0/47: dread fd [0,4194304] 0 2024-03-11T19:55:49.283 INFO:tasks.workunit.client.0.smithi107.stdout:8/33: dwrite f3 [0,4194304] 0 2024-03-11T19:55:49.286 INFO:tasks.workunit.client.0.smithi107.stdout:8/34: mknod ca 0 2024-03-11T19:55:49.290 INFO:tasks.workunit.client.0.smithi107.stdout:4/32: dwrite f2 [0,4194304] 0 2024-03-11T19:55:49.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/21: sync 2024-03-11T19:55:49.299 INFO:tasks.workunit.client.0.smithi107.stdout:9/26: read f1 [3577738,128077] 0 2024-03-11T19:55:49.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/22: creat d1/f4 x:0 0 0 2024-03-11T19:55:49.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/23: symlink d1/l5 0 2024-03-11T19:55:49.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/24: chown d1/l2 161203523 1 2024-03-11T19:55:49.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/25: read - d1/f4 zero size 2024-03-11T19:55:49.303 INFO:tasks.workunit.client.0.smithi107.stdout:2/26: creat d1/f6 x:0 0 0 2024-03-11T19:55:49.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/27: link d1/l5 d1/l7 0 2024-03-11T19:55:49.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/28: dread - d1/f4 zero size 2024-03-11T19:55:49.305 INFO:tasks.workunit.client.0.smithi107.stdout:2/29: unlink d1/f4 0 2024-03-11T19:55:49.307 INFO:tasks.workunit.client.0.smithi107.stdout:0/48: write fd [3836753,129310] 0 2024-03-11T19:55:49.307 INFO:tasks.workunit.client.0.smithi107.stdout:2/30: rename d1/l7 to d1/l8 0 2024-03-11T19:55:49.307 INFO:tasks.workunit.client.0.smithi107.stdout:2/31: write d1/f6 [707826,105738] 0 2024-03-11T19:55:49.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/31: dread f0 [0,4194304] 0 2024-03-11T19:55:49.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/32: truncate f5 1174307 0 2024-03-11T19:55:49.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/32: dread f4 [0,4194304] 0 2024-03-11T19:55:49.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/33: mknod c7 0 2024-03-11T19:55:49.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/33: creat f9 x:0 0 0 2024-03-11T19:55:49.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/34: write f9 [486335,49343] 0 2024-03-11T19:55:49.316 INFO:tasks.workunit.client.0.smithi107.stdout:4/33: write f2 [3369833,39973] 0 2024-03-11T19:55:49.316 INFO:tasks.workunit.client.0.smithi107.stdout:4/34: chown f2 151 1 2024-03-11T19:55:49.316 INFO:tasks.workunit.client.0.smithi107.stdout:2/32: link d1/l8 d1/l9 0 2024-03-11T19:55:49.317 INFO:tasks.workunit.client.0.smithi107.stdout:6/34: getdents . 0 2024-03-11T19:55:49.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/49: dread f3 [0,4194304] 0 2024-03-11T19:55:49.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/50: readlink l1 0 2024-03-11T19:55:49.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/33: mknod d1/ca 0 2024-03-11T19:55:49.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/34: chown d1/c3 0 1 2024-03-11T19:55:49.319 INFO:tasks.workunit.client.0.smithi107.stdout:6/35: unlink f6 0 2024-03-11T19:55:49.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/33: write f5 [200212,62401] 0 2024-03-11T19:55:49.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/35: write f3 [610172,52976] 0 2024-03-11T19:55:49.323 INFO:tasks.workunit.client.0.smithi107.stdout:2/35: rmdir d1 39 2024-03-11T19:55:49.325 INFO:tasks.workunit.client.0.smithi107.stdout:1/34: symlink d6/la 0 2024-03-11T19:55:49.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/36: read d1/f6 [652391,105167] 0 2024-03-11T19:55:49.328 INFO:tasks.workunit.client.0.smithi107.stdout:2/37: unlink d1/l8 0 2024-03-11T19:55:49.328 INFO:tasks.workunit.client.0.smithi107.stdout:2/38: write d1/f6 [1424915,63194] 0 2024-03-11T19:55:49.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/39: getdents d1 0 2024-03-11T19:55:49.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/27: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/36: dread f3 [0,4194304] 0 2024-03-11T19:55:49.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/28: mkdir d4/d9 0 2024-03-11T19:55:49.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/29: creat d4/d9/fa x:0 0 0 2024-03-11T19:55:49.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/30: readlink l3 0 2024-03-11T19:55:49.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/31: symlink d4/d9/lb 0 2024-03-11T19:55:49.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/32: symlink d4/lc 0 2024-03-11T19:55:49.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/33: symlink d4/ld 0 2024-03-11T19:55:49.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/47: sync 2024-03-11T19:55:49.351 INFO:tasks.workunit.client.0.smithi107.stdout:0/51: dread fd [0,4194304] 0 2024-03-11T19:55:49.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/48: creat d6/fb x:0 0 0 2024-03-11T19:55:49.361 INFO:tasks.workunit.client.0.smithi107.stdout:7/49: mknod d6/cc 0 2024-03-11T19:55:49.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/50: write f5 [4505638,5780] 0 2024-03-11T19:55:49.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/51: fdatasync d6/d9/f8 0 2024-03-11T19:55:49.367 INFO:tasks.workunit.client.0.smithi107.stdout:7/52: mkdir d6/d9/dd 0 2024-03-11T19:55:49.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/53: chown d6/d9 1741 1 2024-03-11T19:55:49.373 INFO:tasks.workunit.client.0.smithi107.stdout:7/54: chown d6/cc 3 1 2024-03-11T19:55:49.373 INFO:tasks.workunit.client.0.smithi107.stdout:7/55: dread - d6/d9/f8 zero size 2024-03-11T19:55:49.375 INFO:tasks.workunit.client.0.smithi107.stdout:7/56: link d6/fb d6/fe 0 2024-03-11T19:55:49.375 INFO:tasks.workunit.client.0.smithi107.stdout:7/57: creat d6/ff x:0 0 0 2024-03-11T19:55:49.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/58: truncate f5 4183282 0 2024-03-11T19:55:49.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/59: chown d6/cc 57182 1 2024-03-11T19:55:49.384 INFO:tasks.workunit.client.0.smithi107.stdout:5/35: dwrite f9 [0,4194304] 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:5/36: truncate f9 4823256 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/60: unlink f5 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/61: truncate d6/fb 648582 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/62: write d6/fb [908693,107020] 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/37: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/38: readlink d4/d7/l9 0 2024-03-11T19:55:49.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/39: mkdir d4/d7/db 0 2024-03-11T19:55:49.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/40: creat d4/fc x:0 0 0 2024-03-11T19:55:49.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/41: symlink d4/ld 0 2024-03-11T19:55:49.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/42: readlink d4/d7/l9 0 2024-03-11T19:55:49.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/34: dwrite f1 [0,4194304] 0 2024-03-11T19:55:49.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/35: creat d4/fe x:0 0 0 2024-03-11T19:55:49.401 INFO:tasks.workunit.client.0.smithi107.stdout:0/52: dwrite f3 [0,4194304] 0 2024-03-11T19:55:49.403 INFO:tasks.workunit.client.0.smithi107.stdout:0/53: creat de/f13 x:0 0 0 2024-03-11T19:55:49.403 INFO:tasks.workunit.client.0.smithi107.stdout:0/54: truncate de/f13 24513 0 2024-03-11T19:55:49.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/63: dread d6/fe [0,4194304] 0 2024-03-11T19:55:49.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/64: rename d6 to d6/d9/d10 22 2024-03-11T19:55:49.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/65: dread - d6/d9/f8 zero size 2024-03-11T19:55:49.425 INFO:tasks.workunit.client.0.smithi107.stdout:6/36: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.427 INFO:tasks.workunit.client.0.smithi107.stdout:6/37: rename f2 to f8 0 2024-03-11T19:55:49.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/36: dread f1 [0,4194304] 0 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/37: chown d4/d9 526 1 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/38: read - d4/d9/fa zero size 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/39: dread - d4/f7 zero size 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/40: readlink l3 0 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/41: read - d4/d9/fa zero size 2024-03-11T19:55:49.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/42: chown d4 5399 1 2024-03-11T19:55:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/38: truncate f0 2122050 0 2024-03-11T19:55:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/39: dread - f3 zero size 2024-03-11T19:55:49.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/37: write f5 [571401,105180] 0 2024-03-11T19:55:49.447 INFO:tasks.workunit.client.0.smithi107.stdout:2/40: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:55:49.450 INFO:tasks.workunit.client.0.smithi107.stdout:0/55: dwrite f5 [0,4194304] 0 2024-03-11T19:55:49.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/38: truncate f4 2232088 0 2024-03-11T19:55:49.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/39: readlink l7 0 2024-03-11T19:55:49.450 INFO:tasks.workunit.client.0.smithi107.stdout:2/41: mknod d1/cb 0 2024-03-11T19:55:49.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/43: dwrite d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:49.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/56: mknod de/d11/c14 0 2024-03-11T19:55:49.453 INFO:tasks.workunit.client.0.smithi107.stdout:5/40: write f4 [1098615,116108] 0 2024-03-11T19:55:49.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/42: creat d1/fc x:0 0 0 2024-03-11T19:55:49.457 INFO:tasks.workunit.client.0.smithi107.stdout:0/57: mknod de/c15 0 2024-03-11T19:55:49.476 INFO:tasks.workunit.client.0.smithi107.stdout:3/60: sync 2024-03-11T19:55:49.476 INFO:tasks.workunit.client.0.smithi107.stdout:7/66: dwrite d6/fb [0,4194304] 0 2024-03-11T19:55:49.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/40: dwrite f3 [0,4194304] 0 2024-03-11T19:55:49.483 INFO:tasks.workunit.client.0.smithi107.stdout:7/67: creat d6/d9/f11 x:0 0 0 2024-03-11T19:55:49.483 INFO:tasks.workunit.client.0.smithi107.stdout:4/44: dread f0 [0,4194304] 0 2024-03-11T19:55:49.483 INFO:tasks.workunit.client.0.smithi107.stdout:3/61: dread d1/fa [0,4194304] 0 2024-03-11T19:55:49.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/41: creat f9 x:0 0 0 2024-03-11T19:55:49.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/42: readlink - no filename 2024-03-11T19:55:49.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/43: readlink - no filename 2024-03-11T19:55:49.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/44: creat fa x:0 0 0 2024-03-11T19:55:49.485 INFO:tasks.workunit.client.0.smithi107.stdout:5/41: write f9 [14809,124710] 0 2024-03-11T19:55:49.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/45: mkdir d4/d7/de 0 2024-03-11T19:55:49.489 INFO:tasks.workunit.client.0.smithi107.stdout:5/42: mkdir da 0 2024-03-11T19:55:49.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/46: symlink d4/d7/db/lf 0 2024-03-11T19:55:49.491 INFO:tasks.workunit.client.0.smithi107.stdout:7/68: getdents d6/d9 0 2024-03-11T19:55:49.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/47: truncate d4/d7/f8 555948 0 2024-03-11T19:55:49.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/69: mknod d6/d9/dd/c12 0 2024-03-11T19:55:49.496 INFO:tasks.workunit.client.0.smithi107.stdout:4/48: rmdir d4 39 2024-03-11T19:55:49.496 INFO:tasks.workunit.client.0.smithi107.stdout:4/49: write f3 [1296382,74952] 0 2024-03-11T19:55:49.497 INFO:tasks.workunit.client.0.smithi107.stdout:7/70: rename d6/d9/dd to d6/d9/d13 0 2024-03-11T19:55:49.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/62: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:49.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/63: mkdir d1/d10/d13 0 2024-03-11T19:55:49.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/64: chown d1/ld 264 1 2024-03-11T19:55:49.512 INFO:tasks.workunit.client.0.smithi107.stdout:3/65: dread d1/fa [0,4194304] 0 2024-03-11T19:55:49.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/66: rmdir d1/d10 39 2024-03-11T19:55:49.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/67: write d1/fa [1269590,54450] 0 2024-03-11T19:55:49.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/68: fsync d1/fa 0 2024-03-11T19:55:49.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/69: stat d1/d10/d13 0 2024-03-11T19:55:49.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/43: write f9 [4256871,7952] 0 2024-03-11T19:55:49.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/44: chown da 3389415 1 2024-03-11T19:55:49.522 INFO:tasks.workunit.client.0.smithi107.stdout:3/70: creat d1/d12/f14 x:0 0 0 2024-03-11T19:55:49.525 INFO:tasks.workunit.client.0.smithi107.stdout:5/45: creat da/fb x:0 0 0 2024-03-11T19:55:49.525 INFO:tasks.workunit.client.0.smithi107.stdout:5/46: creat da/fc x:0 0 0 2024-03-11T19:55:49.529 INFO:tasks.workunit.client.0.smithi107.stdout:3/71: link d1/l9 d1/d10/l15 0 2024-03-11T19:55:49.529 INFO:tasks.workunit.client.0.smithi107.stdout:5/47: mkdir da/dd 0 2024-03-11T19:55:49.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/45: dwrite f9 [0,4194304] 0 2024-03-11T19:55:49.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/46: readlink - no filename 2024-03-11T19:55:49.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/47: chown f0 709705 1 2024-03-11T19:55:49.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/48: rmdir da 39 2024-03-11T19:55:49.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/49: rename c8 to da/dd/ce 0 2024-03-11T19:55:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/50: symlink da/dd/lf 0 2024-03-11T19:55:49.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/51: rmdir da 39 2024-03-11T19:55:49.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/52: dread - da/fc zero size 2024-03-11T19:55:49.547 INFO:tasks.workunit.client.0.smithi107.stdout:5/53: mknod da/dd/c10 0 2024-03-11T19:55:49.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/54: link f4 da/dd/f11 0 2024-03-11T19:55:49.550 INFO:tasks.workunit.client.0.smithi107.stdout:5/55: creat da/dd/f12 x:0 0 0 2024-03-11T19:55:49.550 INFO:tasks.workunit.client.0.smithi107.stdout:5/56: chown f9 976238633 1 2024-03-11T19:55:49.552 INFO:tasks.workunit.client.0.smithi107.stdout:4/50: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.554 INFO:tasks.workunit.client.0.smithi107.stdout:6/48: write f0 [1588035,70484] 0 2024-03-11T19:55:49.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/57: stat l7 0 2024-03-11T19:55:49.554 INFO:tasks.workunit.client.0.smithi107.stdout:7/71: dwrite d6/ff [0,4194304] 0 2024-03-11T19:55:49.555 INFO:tasks.workunit.client.0.smithi107.stdout:4/51: symlink d4/d7/db/l10 0 2024-03-11T19:55:49.555 INFO:tasks.workunit.client.0.smithi107.stdout:4/52: truncate d4/d7/f8 1230358 0 2024-03-11T19:55:49.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/49: symlink lb 0 2024-03-11T19:55:49.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/50: rename f9 to fc 0 2024-03-11T19:55:49.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/72: dwrite d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:49.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/72: truncate d6/ff 3084173 0 2024-03-11T19:55:49.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/51: getdents . 0 2024-03-11T19:55:49.573 INFO:tasks.workunit.client.0.smithi107.stdout:7/73: creat d6/d9/d13/f14 x:0 0 0 2024-03-11T19:55:49.574 INFO:tasks.workunit.client.0.smithi107.stdout:4/53: truncate f0 2666898 0 2024-03-11T19:55:49.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/35: sync 2024-03-11T19:55:49.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/36: mknod cb 0 2024-03-11T19:55:49.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/37: write f4 [1502513,64416] 0 2024-03-11T19:55:49.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/38: chown ca 64338 1 2024-03-11T19:55:49.589 INFO:tasks.workunit.client.0.smithi107.stdout:8/39: rename ca to cc 0 2024-03-11T19:55:49.590 INFO:tasks.workunit.client.0.smithi107.stdout:8/40: mkdir dd 0 2024-03-11T19:55:49.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/73: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:49.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/58: dwrite da/dd/f12 [0,4194304] 0 2024-03-11T19:55:49.592 INFO:tasks.workunit.client.0.smithi107.stdout:3/74: mkdir d1/d10/d13/d16 0 2024-03-11T19:55:49.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/59: rmdir da/dd 39 2024-03-11T19:55:49.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/60: write da/fb [997351,64405] 0 2024-03-11T19:55:49.601 INFO:tasks.workunit.client.0.smithi107.stdout:5/61: creat da/dd/f13 x:0 0 0 2024-03-11T19:55:49.602 INFO:tasks.workunit.client.0.smithi107.stdout:6/52: dread fc [0,4194304] 0 2024-03-11T19:55:49.606 INFO:tasks.workunit.client.0.smithi107.stdout:4/54: dread f0 [0,4194304] 0 2024-03-11T19:55:49.611 INFO:tasks.workunit.client.0.smithi107.stdout:4/55: read f3 [1059953,55160] 0 2024-03-11T19:55:49.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/56: rename d4/ld to d4/l11 0 2024-03-11T19:55:49.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/57: fsync f2 0 2024-03-11T19:55:49.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/58: mknod d4/d7/db/c12 0 2024-03-11T19:55:49.620 INFO:tasks.workunit.client.0.smithi107.stdout:3/75: write d1/f5 [7564180,115646] 0 2024-03-11T19:55:49.623 INFO:tasks.workunit.client.0.smithi107.stdout:3/76: mkdir d1/d10/d13/d16/d17 0 2024-03-11T19:55:49.623 INFO:tasks.workunit.client.0.smithi107.stdout:8/41: dread f3 [0,4194304] 0 2024-03-11T19:55:49.626 INFO:tasks.workunit.client.0.smithi107.stdout:4/59: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:49.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/60: rmdir d4 39 2024-03-11T19:55:49.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/61: mknod d4/c13 0 2024-03-11T19:55:49.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/62: dread - d4/fc zero size 2024-03-11T19:55:49.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/63: dread - d4/f6 zero size 2024-03-11T19:55:49.641 INFO:tasks.workunit.client.0.smithi107.stdout:6/53: dwrite f5 [0,4194304] 0 2024-03-11T19:55:49.645 INFO:tasks.workunit.client.0.smithi107.stdout:4/64: dread f3 [0,4194304] 0 2024-03-11T19:55:49.645 INFO:tasks.workunit.client.0.smithi107.stdout:6/54: creat fd x:0 0 0 2024-03-11T19:55:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/55: creat fe x:0 0 0 2024-03-11T19:55:49.646 INFO:tasks.workunit.client.0.smithi107.stdout:4/65: unlink f2 0 2024-03-11T19:55:49.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/66: write d4/fc [954762,103778] 0 2024-03-11T19:55:49.648 INFO:tasks.workunit.client.0.smithi107.stdout:6/56: mknod cf 0 2024-03-11T19:55:49.648 INFO:tasks.workunit.client.0.smithi107.stdout:4/67: symlink d4/d7/db/l14 0 2024-03-11T19:55:49.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/57: mkdir d10 0 2024-03-11T19:55:49.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/58: symlink d10/l11 0 2024-03-11T19:55:49.651 INFO:tasks.workunit.client.0.smithi107.stdout:6/59: mknod d10/c12 0 2024-03-11T19:55:49.651 INFO:tasks.workunit.client.0.smithi107.stdout:6/60: truncate fd 892778 0 2024-03-11T19:55:49.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/77: dwrite d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:49.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/78: mknod d1/d12/c18 0 2024-03-11T19:55:49.662 INFO:tasks.workunit.client.0.smithi107.stdout:3/79: stat d1/d12 0 2024-03-11T19:55:49.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/80: mkdir d1/d19 0 2024-03-11T19:55:49.666 INFO:tasks.workunit.client.0.smithi107.stdout:9/43: sync 2024-03-11T19:55:49.666 INFO:tasks.workunit.client.0.smithi107.stdout:9/44: stat d4/d9/fa 0 2024-03-11T19:55:49.667 INFO:tasks.workunit.client.0.smithi107.stdout:1/35: sync 2024-03-11T19:55:49.667 INFO:tasks.workunit.client.0.smithi107.stdout:3/81: mkdir d1/d12/d1a 0 2024-03-11T19:55:49.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/62: dwrite f4 [0,4194304] 0 2024-03-11T19:55:49.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/63: stat l7 0 2024-03-11T19:55:49.672 INFO:tasks.workunit.client.0.smithi107.stdout:9/45: unlink d4/f7 0 2024-03-11T19:55:49.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/64: unlink da/dd/f12 0 2024-03-11T19:55:49.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/65: unlink da/dd/lf 0 2024-03-11T19:55:49.675 INFO:tasks.workunit.client.0.smithi107.stdout:5/66: unlink da/dd/c10 0 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/67: mkdir da/d14 0 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/68: chown da/dd 458134 1 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/69: fdatasync f4 0 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/70: chown f9 869987 1 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/71: write f9 [2846408,80633] 0 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/72: chown f4 48280945 1 2024-03-11T19:55:49.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/73: chown da/dd/f11 1492147643 1 2024-03-11T19:55:49.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/74: symlink da/d14/l15 0 2024-03-11T19:55:49.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/75: fsync da/dd/f13 0 2024-03-11T19:55:49.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/76: write f5 [2790760,6924] 0 2024-03-11T19:55:49.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/77: chown da 86887910 1 2024-03-11T19:55:49.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/78: creat da/f16 x:0 0 0 2024-03-11T19:55:49.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/79: mknod da/d14/c17 0 2024-03-11T19:55:49.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/80: write da/f16 [507878,24360] 0 2024-03-11T19:55:49.686 INFO:tasks.workunit.client.0.smithi107.stdout:3/82: dread d1/fa [0,4194304] 0 2024-03-11T19:55:49.689 INFO:tasks.workunit.client.0.smithi107.stdout:3/83: mknod d1/d10/c1b 0 2024-03-11T19:55:49.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/84: truncate d1/fa 1940081 0 2024-03-11T19:55:49.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/61: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.699 INFO:tasks.workunit.client.0.smithi107.stdout:6/62: getdents d10 0 2024-03-11T19:55:49.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/63: chown cf 1452881118 1 2024-03-11T19:55:49.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/64: fdatasync f8 0 2024-03-11T19:55:49.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/81: write da/fb [45708,63443] 0 2024-03-11T19:55:49.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/82: write da/fc [558569,18891] 0 2024-03-11T19:55:49.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/83: write f9 [5081306,58026] 0 2024-03-11T19:55:49.705 INFO:tasks.workunit.client.0.smithi107.stdout:8/42: dwrite f4 [0,4194304] 0 2024-03-11T19:55:49.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/84: mkdir da/d14/d18 0 2024-03-11T19:55:49.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/43: mkdir dd/de 0 2024-03-11T19:55:49.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/85: unlink da/d14/l15 0 2024-03-11T19:55:49.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/86: read f9 [4190562,44357] 0 2024-03-11T19:55:49.715 INFO:tasks.workunit.client.0.smithi107.stdout:6/65: dread fd [0,4194304] 0 2024-03-11T19:55:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:1/36: dwrite f5 [0,4194304] 0 2024-03-11T19:55:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/87: mkdir da/d14/d19 0 2024-03-11T19:55:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:6/66: rename f5 to d10/f13 0 2024-03-11T19:55:49.721 INFO:tasks.workunit.client.0.smithi107.stdout:6/67: fdatasync fc 0 2024-03-11T19:55:49.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/68: chown d10/f13 47 1 2024-03-11T19:55:49.734 INFO:tasks.workunit.client.0.smithi107.stdout:9/46: dwrite f0 [0,4194304] 0 2024-03-11T19:55:49.734 INFO:tasks.workunit.client.0.smithi107.stdout:9/47: chown d4 292859 1 2024-03-11T19:55:49.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/88: read da/f16 [174249,97432] 0 2024-03-11T19:55:49.744 INFO:tasks.workunit.client.0.smithi107.stdout:5/89: unlink da/fc 0 2024-03-11T19:55:49.744 INFO:tasks.workunit.client.0.smithi107.stdout:5/90: creat da/f1a x:0 0 0 2024-03-11T19:55:49.745 INFO:tasks.workunit.client.0.smithi107.stdout:3/85: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:49.747 INFO:tasks.workunit.client.0.smithi107.stdout:5/91: link da/f1a da/f1b 0 2024-03-11T19:55:49.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/92: creat da/d14/d18/f1c x:0 0 0 2024-03-11T19:55:49.749 INFO:tasks.workunit.client.0.smithi107.stdout:5/93: readlink l7 0 2024-03-11T19:55:49.749 INFO:tasks.workunit.client.0.smithi107.stdout:5/94: creat da/d14/d19/f1d x:0 0 0 2024-03-11T19:55:49.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/68: dwrite d4/fc [0,4194304] 0 2024-03-11T19:55:49.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/69: truncate d4/f6 554220 0 2024-03-11T19:55:49.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/70: write f3 [948609,114849] 0 2024-03-11T19:55:49.766 INFO:tasks.workunit.client.0.smithi107.stdout:1/37: dwrite f4 [0,4194304] 0 2024-03-11T19:55:49.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/48: dread f0 [0,4194304] 0 2024-03-11T19:55:49.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/49: read - d4/fe zero size 2024-03-11T19:55:49.774 INFO:tasks.workunit.client.0.smithi107.stdout:9/50: truncate d4/d9/fa 879499 0 2024-03-11T19:55:49.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/71: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:49.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/72: stat d4/fc 0 2024-03-11T19:55:49.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/73: readlink d4/d7/db/lf 0 2024-03-11T19:55:49.780 INFO:tasks.workunit.client.0.smithi107.stdout:9/51: dread d4/d9/fa [0,4194304] 0 2024-03-11T19:55:49.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/52: dread - d4/fe zero size 2024-03-11T19:55:49.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/69: dwrite fd [0,4194304] 0 2024-03-11T19:55:49.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/43: sync 2024-03-11T19:55:49.787 INFO:tasks.workunit.client.0.smithi107.stdout:4/74: dread f0 [0,4194304] 0 2024-03-11T19:55:49.789 INFO:tasks.workunit.client.0.smithi107.stdout:1/38: link d6/la d6/lb 0 2024-03-11T19:55:49.790 INFO:tasks.workunit.client.0.smithi107.stdout:9/53: link l3 d4/lf 0 2024-03-11T19:55:49.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/58: sync 2024-03-11T19:55:49.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/39: creat d6/fc x:0 0 0 2024-03-11T19:55:49.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/70: creat d10/f14 x:0 0 0 2024-03-11T19:55:49.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/74: sync 2024-03-11T19:55:49.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/75: chown d6/ff 11 1 2024-03-11T19:55:49.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/76: write d6/d9/d13/f14 [556791,110713] 0 2024-03-11T19:55:49.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/77: fdatasync d6/ff 0 2024-03-11T19:55:49.792 INFO:tasks.workunit.client.0.smithi107.stdout:2/44: rename d1/c3 to d1/cd 0 2024-03-11T19:55:49.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/75: unlink d4/d7/db/l14 0 2024-03-11T19:55:49.794 INFO:tasks.workunit.client.0.smithi107.stdout:1/40: unlink l0 0 2024-03-11T19:55:49.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/78: link d6/fb d6/da/f15 0 2024-03-11T19:55:49.796 INFO:tasks.workunit.client.0.smithi107.stdout:4/76: creat d4/d7/de/f15 x:0 0 0 2024-03-11T19:55:49.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/79: mkdir d6/da/d16 0 2024-03-11T19:55:49.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/80: chown d6/fb 5664 1 2024-03-11T19:55:49.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/81: readlink - no filename 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/41: chown d6/f9 5 1 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/42: dread - d6/fc zero size 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/43: fdatasync d6/f9 0 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/45: getdents d1 0 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/46: mknod d1/ce 0 2024-03-11T19:55:49.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/47: read - d1/fc zero size 2024-03-11T19:55:49.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/86: dwrite d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:49.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/48: link d1/ca d1/cf 0 2024-03-11T19:55:49.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/95: dwrite da/fb [0,4194304] 0 2024-03-11T19:55:49.809 INFO:tasks.workunit.client.0.smithi107.stdout:3/87: rename d1/l9 to d1/d10/l1c 0 2024-03-11T19:55:49.815 INFO:tasks.workunit.client.0.smithi107.stdout:0/59: write f5 [16460,109270] 0 2024-03-11T19:55:49.816 INFO:tasks.workunit.client.0.smithi107.stdout:9/54: write f0 [3606959,84508] 0 2024-03-11T19:55:49.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/88: rmdir d1/d10/d13 39 2024-03-11T19:55:49.818 INFO:tasks.workunit.client.0.smithi107.stdout:3/89: write d1/f5 [6581678,66590] 0 2024-03-11T19:55:49.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/49: getdents d1 0 2024-03-11T19:55:49.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/55: mkdir d4/d10 0 2024-03-11T19:55:49.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/56: creat d4/f11 x:0 0 0 2024-03-11T19:55:49.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/50: creat d1/f10 x:0 0 0 2024-03-11T19:55:49.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/51: write d1/f10 [953683,45427] 0 2024-03-11T19:55:49.822 INFO:tasks.workunit.client.0.smithi107.stdout:9/57: rename f0 to d4/f12 0 2024-03-11T19:55:49.824 INFO:tasks.workunit.client.0.smithi107.stdout:9/58: mknod d4/c13 0 2024-03-11T19:55:49.825 INFO:tasks.workunit.client.0.smithi107.stdout:9/59: creat d4/d9/f14 x:0 0 0 2024-03-11T19:55:49.825 INFO:tasks.workunit.client.0.smithi107.stdout:9/60: creat d4/d9/f15 x:0 0 0 2024-03-11T19:55:49.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/52: getdents d1 0 2024-03-11T19:55:49.829 INFO:tasks.workunit.client.0.smithi107.stdout:6/71: write f8 [836694,45696] 0 2024-03-11T19:55:49.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/53: rename d1 to d1/d11 22 2024-03-11T19:55:49.830 INFO:tasks.workunit.client.0.smithi107.stdout:4/77: write d4/fc [2718333,9953] 0 2024-03-11T19:55:49.831 INFO:tasks.workunit.client.0.smithi107.stdout:4/78: truncate d4/d7/de/f15 335574 0 2024-03-11T19:55:49.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/72: symlink d10/l15 0 2024-03-11T19:55:49.831 INFO:tasks.workunit.client.0.smithi107.stdout:4/79: stat d4/l11 0 2024-03-11T19:55:49.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/73: unlink d10/l15 0 2024-03-11T19:55:49.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/74: write f0 [4499480,113851] 0 2024-03-11T19:55:49.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/75: truncate fc 4450770 0 2024-03-11T19:55:49.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/76: write d10/f14 [713514,16256] 0 2024-03-11T19:55:49.834 INFO:tasks.workunit.client.0.smithi107.stdout:4/80: symlink d4/d7/de/l16 0 2024-03-11T19:55:49.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/77: mkdir d10/d16 0 2024-03-11T19:55:49.839 INFO:tasks.workunit.client.0.smithi107.stdout:7/82: write d6/fb [741671,21653] 0 2024-03-11T19:55:49.839 INFO:tasks.workunit.client.0.smithi107.stdout:7/83: write d6/ff [3471159,115088] 0 2024-03-11T19:55:49.853 INFO:tasks.workunit.client.0.smithi107.stdout:0/60: dread f3 [0,4194304] 0 2024-03-11T19:55:49.853 INFO:tasks.workunit.client.0.smithi107.stdout:0/61: creat de/f16 x:0 0 0 2024-03-11T19:55:49.853 INFO:tasks.workunit.client.0.smithi107.stdout:0/62: creat de/f17 x:0 0 0 2024-03-11T19:55:49.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/63: creat de/f18 x:0 0 0 2024-03-11T19:55:49.856 INFO:tasks.workunit.client.0.smithi107.stdout:4/81: dread f3 [0,4194304] 0 2024-03-11T19:55:49.856 INFO:tasks.workunit.client.0.smithi107.stdout:4/82: chown d4/l11 7824072 1 2024-03-11T19:55:49.859 INFO:tasks.workunit.client.0.smithi107.stdout:4/83: write d4/f6 [160319,45057] 0 2024-03-11T19:55:49.861 INFO:tasks.workunit.client.0.smithi107.stdout:4/84: creat d4/d7/f17 x:0 0 0 2024-03-11T19:55:49.862 INFO:tasks.workunit.client.0.smithi107.stdout:4/85: write f3 [2025088,17268] 0 2024-03-11T19:55:49.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/54: dread d1/f10 [0,4194304] 0 2024-03-11T19:55:49.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/55: readlink d1/l9 0 2024-03-11T19:55:49.864 INFO:tasks.workunit.client.0.smithi107.stdout:4/86: symlink d4/l18 0 2024-03-11T19:55:49.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/84: dread d6/d9/d13/f14 [0,4194304] 0 2024-03-11T19:55:49.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/85: write d6/fb [2350394,101612] 0 2024-03-11T19:55:49.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/86: chown d6 29772109 1 2024-03-11T19:55:49.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/96: dwrite f4 [0,4194304] 0 2024-03-11T19:55:49.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/56: read d1/f10 [157262,75900] 0 2024-03-11T19:55:49.874 INFO:tasks.workunit.client.0.smithi107.stdout:2/57: link d1/cd d1/c12 0 2024-03-11T19:55:49.875 INFO:tasks.workunit.client.0.smithi107.stdout:2/58: write d1/fc [1039365,14600] 0 2024-03-11T19:55:49.883 INFO:tasks.workunit.client.0.smithi107.stdout:4/87: dread f0 [0,4194304] 0 2024-03-11T19:55:49.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/88: dread d4/d7/de/f15 [0,4194304] 0 2024-03-11T19:55:49.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/44: dwrite d6/f9 [0,4194304] 0 2024-03-11T19:55:49.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/45: write d6/fc [7381,7289] 0 2024-03-11T19:55:49.897 INFO:tasks.workunit.client.0.smithi107.stdout:1/46: creat d6/fd x:0 0 0 2024-03-11T19:55:49.899 INFO:tasks.workunit.client.0.smithi107.stdout:4/89: mkdir d4/d7/de/d19 0 2024-03-11T19:55:49.900 INFO:tasks.workunit.client.0.smithi107.stdout:1/47: symlink d6/le 0 2024-03-11T19:55:49.900 INFO:tasks.workunit.client.0.smithi107.stdout:7/87: dread d6/fb [0,4194304] 0 2024-03-11T19:55:49.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/90: mknod d4/c1a 0 2024-03-11T19:55:49.902 INFO:tasks.workunit.client.0.smithi107.stdout:1/48: symlink d6/lf 0 2024-03-11T19:55:49.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/91: rename d4/l18 to d4/d7/l1b 0 2024-03-11T19:55:49.904 INFO:tasks.workunit.client.0.smithi107.stdout:4/92: read d4/f6 [182935,101533] 0 2024-03-11T19:55:49.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/88: write d6/d9/d13/f14 [352448,110578] 0 2024-03-11T19:55:49.908 INFO:tasks.workunit.client.0.smithi107.stdout:8/44: sync 2024-03-11T19:55:49.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/97: write da/dd/f11 [1385757,5274] 0 2024-03-11T19:55:49.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/93: rename d4/d7/l1b to d4/d7/de/d19/l1c 0 2024-03-11T19:55:49.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/45: truncate f3 3016060 0 2024-03-11T19:55:49.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/46: readlink l9 0 2024-03-11T19:55:49.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/64: dwrite fd [4194304,4194304] 0 2024-03-11T19:55:49.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/65: unlink ca 0 2024-03-11T19:55:49.918 INFO:tasks.workunit.client.0.smithi107.stdout:5/98: dread da/f16 [0,4194304] 0 2024-03-11T19:55:49.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/99: mkdir da/d14/d18/d1e 0 2024-03-11T19:55:49.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/94: dread f0 [0,4194304] 0 2024-03-11T19:55:49.938 INFO:tasks.workunit.client.0.smithi107.stdout:4/95: truncate f3 1524702 0 2024-03-11T19:55:49.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/59: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:49.954 INFO:tasks.workunit.client.0.smithi107.stdout:3/90: dwrite d1/f5 [4194304,4194304] 0 2024-03-11T19:55:49.956 INFO:tasks.workunit.client.0.smithi107.stdout:2/60: rename d1/cb to d1/c13 0 2024-03-11T19:55:49.958 INFO:tasks.workunit.client.0.smithi107.stdout:3/91: link d1/d10/c11 d1/d10/d13/c1d 0 2024-03-11T19:55:49.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/66: dwrite fd [4194304,4194304] 0 2024-03-11T19:55:49.960 INFO:tasks.workunit.client.0.smithi107.stdout:0/67: symlink de/d11/l19 0 2024-03-11T19:55:49.960 INFO:tasks.workunit.client.0.smithi107.stdout:0/68: fsync f5 0 2024-03-11T19:55:49.961 INFO:tasks.workunit.client.0.smithi107.stdout:7/89: dwrite d6/fb [0,4194304] 0 2024-03-11T19:55:49.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/47: dwrite f3 [0,4194304] 0 2024-03-11T19:55:49.963 INFO:tasks.workunit.client.0.smithi107.stdout:4/96: dread d4/f6 [0,4194304] 0 2024-03-11T19:55:49.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/69: creat de/d11/f1a x:0 0 0 2024-03-11T19:55:49.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/70: chown de/f17 2367429 1 2024-03-11T19:55:49.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/71: dread - de/f18 zero size 2024-03-11T19:55:49.968 INFO:tasks.workunit.client.0.smithi107.stdout:7/90: rename d6/da/f15 to d6/f17 0 2024-03-11T19:55:49.969 INFO:tasks.workunit.client.0.smithi107.stdout:0/72: creat de/d11/f1b x:0 0 0 2024-03-11T19:55:49.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/49: write f5 [1743172,44633] 0 2024-03-11T19:55:49.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/50: creat d6/f10 x:0 0 0 2024-03-11T19:55:49.979 INFO:tasks.workunit.client.0.smithi107.stdout:1/51: fsync d6/fd 0 2024-03-11T19:55:49.980 INFO:tasks.workunit.client.0.smithi107.stdout:6/78: sync 2024-03-11T19:55:49.980 INFO:tasks.workunit.client.0.smithi107.stdout:9/61: sync 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/62: creat d4/f16 x:0 0 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/63: write d4/f11 [926415,49275] 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/64: getdents d4/d10 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/65: write d4/f12 [1316821,224] 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/66: write d4/f11 [1662836,100233] 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/67: mkdir d4/d10/d17 0 2024-03-11T19:55:49.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/68: rename d4/d10 to d4/d10/d18 22 2024-03-11T19:55:49.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/69: creat d4/d10/d17/f19 x:0 0 0 2024-03-11T19:55:49.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/70: fdatasync d4/d10/d17/f19 0 2024-03-11T19:55:49.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/71: creat d4/d10/d17/f1a x:0 0 0 2024-03-11T19:55:49.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/100: dwrite da/fb [0,4194304] 0 2024-03-11T19:55:49.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/52: dread d6/fc [0,4194304] 0 2024-03-11T19:55:49.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/53: readlink d6/lf 0 2024-03-11T19:55:49.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/54: chown d6 142021 1 2024-03-11T19:55:49.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/55: truncate d6/fd 72288 0 2024-03-11T19:55:50.001 INFO:tasks.workunit.client.0.smithi107.stdout:2/61: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:50.004 INFO:tasks.workunit.client.0.smithi107.stdout:3/92: read d1/f5 [3476349,17202] 0 2024-03-11T19:55:50.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/93: fsync d1/f5 0 2024-03-11T19:55:50.008 INFO:tasks.workunit.client.0.smithi107.stdout:5/101: rename da/f1b to da/dd/f1f 0 2024-03-11T19:55:50.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/102: read - da/d14/d19/f1d zero size 2024-03-11T19:55:50.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/62: rename d1 to d1/d14 22 2024-03-11T19:55:50.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/103: symlink da/dd/l20 0 2024-03-11T19:55:50.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/104: rename da/f16 to da/dd/f21 0 2024-03-11T19:55:50.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/105: write da/dd/f13 [224849,89075] 0 2024-03-11T19:55:50.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/48: write f3 [1797675,102217] 0 2024-03-11T19:55:50.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/56: symlink d6/l11 0 2024-03-11T19:55:50.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/73: dwrite f5 [0,4194304] 0 2024-03-11T19:55:50.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/74: fsync de/f17 0 2024-03-11T19:55:50.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/57: unlink d6/l11 0 2024-03-11T19:55:50.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/49: symlink dd/de/lf 0 2024-03-11T19:55:50.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/75: symlink de/l1c 0 2024-03-11T19:55:50.030 INFO:tasks.workunit.client.0.smithi107.stdout:7/91: dread d6/d9/d13/f14 [0,4194304] 0 2024-03-11T19:55:50.030 INFO:tasks.workunit.client.0.smithi107.stdout:1/58: rename d6/f9 to d6/f12 0 2024-03-11T19:55:50.030 INFO:tasks.workunit.client.0.smithi107.stdout:1/59: stat d6/fc 0 2024-03-11T19:55:50.035 INFO:tasks.workunit.client.0.smithi107.stdout:0/76: getdents de 0 2024-03-11T19:55:50.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/79: dwrite d10/f14 [0,4194304] 0 2024-03-11T19:55:50.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/80: fdatasync f3 0 2024-03-11T19:55:50.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/92: mknod d6/da/d16/c18 0 2024-03-11T19:55:50.039 INFO:tasks.workunit.client.0.smithi107.stdout:1/60: rename d6/lb to d6/l13 0 2024-03-11T19:55:50.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/77: unlink de/c10 0 2024-03-11T19:55:50.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/81: creat d10/d16/f17 x:0 0 0 2024-03-11T19:55:50.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/82: dread - fa zero size 2024-03-11T19:55:50.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/83: symlink d10/d16/l18 0 2024-03-11T19:55:50.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/84: truncate fe 116931 0 2024-03-11T19:55:50.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/85: truncate d10/d16/f17 265230 0 2024-03-11T19:55:50.046 INFO:tasks.workunit.client.0.smithi107.stdout:6/86: mkdir d10/d19 0 2024-03-11T19:55:50.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/63: dwrite d1/f10 [0,4194304] 0 2024-03-11T19:55:50.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/50: dread f4 [0,4194304] 0 2024-03-11T19:55:50.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/51: chown dd/de 1051078942 1 2024-03-11T19:55:50.071 INFO:tasks.workunit.client.0.smithi107.stdout:7/93: dread d6/f17 [0,4194304] 0 2024-03-11T19:55:50.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/61: getdents d6 0 2024-03-11T19:55:50.073 INFO:tasks.workunit.client.0.smithi107.stdout:9/72: dwrite d4/f12 [4194304,4194304] 0 2024-03-11T19:55:50.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/52: truncate f3 3714276 0 2024-03-11T19:55:50.081 INFO:tasks.workunit.client.0.smithi107.stdout:6/87: write fc [2616340,73178] 0 2024-03-11T19:55:50.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/94: dwrite d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:50.086 INFO:tasks.workunit.client.0.smithi107.stdout:3/95: mknod d1/d19/c1e 0 2024-03-11T19:55:50.088 INFO:tasks.workunit.client.0.smithi107.stdout:3/96: write d1/fa [1644332,22401] 0 2024-03-11T19:55:50.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/97: rmdir d1/d10/d13/d16 39 2024-03-11T19:55:50.101 INFO:tasks.workunit.client.0.smithi107.stdout:2/64: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:50.106 INFO:tasks.workunit.client.0.smithi107.stdout:1/62: rename f5 to d6/f14 0 2024-03-11T19:55:50.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/78: dwrite de/f16 [0,4194304] 0 2024-03-11T19:55:50.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/63: creat d6/f15 x:0 0 0 2024-03-11T19:55:50.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/64: creat d6/f16 x:0 0 0 2024-03-11T19:55:50.111 INFO:tasks.workunit.client.0.smithi107.stdout:1/65: mknod d6/c17 0 2024-03-11T19:55:50.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/66: symlink d6/l18 0 2024-03-11T19:55:50.122 INFO:tasks.workunit.client.0.smithi107.stdout:3/98: dwrite d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:50.123 INFO:tasks.workunit.client.0.smithi107.stdout:3/99: creat d1/d10/d13/d16/d17/f1f x:0 0 0 2024-03-11T19:55:50.124 INFO:tasks.workunit.client.0.smithi107.stdout:3/100: dread - d1/d10/d13/d16/d17/f1f zero size 2024-03-11T19:55:50.124 INFO:tasks.workunit.client.0.smithi107.stdout:3/101: truncate d1/f5 5374197 0 2024-03-11T19:55:50.128 INFO:tasks.workunit.client.0.smithi107.stdout:8/53: write f4 [2680973,113780] 0 2024-03-11T19:55:50.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/73: dwrite d4/d10/d17/f19 [0,4194304] 0 2024-03-11T19:55:50.136 INFO:tasks.workunit.client.0.smithi107.stdout:6/88: dread d10/f13 [0,4194304] 0 2024-03-11T19:55:50.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/102: dread d1/fa [0,4194304] 0 2024-03-11T19:55:50.139 INFO:tasks.workunit.client.0.smithi107.stdout:6/89: link fc d10/f1a 0 2024-03-11T19:55:50.139 INFO:tasks.workunit.client.0.smithi107.stdout:6/90: chown d10/f1a 935 1 2024-03-11T19:55:50.141 INFO:tasks.workunit.client.0.smithi107.stdout:3/103: creat d1/d12/d1a/f20 x:0 0 0 2024-03-11T19:55:50.144 INFO:tasks.workunit.client.0.smithi107.stdout:6/91: symlink d10/d19/l1b 0 2024-03-11T19:55:50.153 INFO:tasks.workunit.client.0.smithi107.stdout:6/92: dread fe [0,4194304] 0 2024-03-11T19:55:50.155 INFO:tasks.workunit.client.0.smithi107.stdout:8/54: dread f3 [0,4194304] 0 2024-03-11T19:55:50.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/67: dwrite d6/f15 [0,4194304] 0 2024-03-11T19:55:50.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/74: write f1 [6268995,17351] 0 2024-03-11T19:55:50.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/75: creat d4/d10/d17/f1b x:0 0 0 2024-03-11T19:55:50.161 INFO:tasks.workunit.client.0.smithi107.stdout:9/76: write d4/d9/f15 [641444,100616] 0 2024-03-11T19:55:50.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/77: getdents d4/d10 0 2024-03-11T19:55:50.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/78: mkdir d4/d1c 0 2024-03-11T19:55:50.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/65: dwrite d1/f10 [0,4194304] 0 2024-03-11T19:55:50.178 INFO:tasks.workunit.client.0.smithi107.stdout:2/66: creat d1/f15 x:0 0 0 2024-03-11T19:55:50.193 INFO:tasks.workunit.client.0.smithi107.stdout:1/68: write d6/f12 [186115,117912] 0 2024-03-11T19:55:50.193 INFO:tasks.workunit.client.0.smithi107.stdout:0/79: dwrite de/d11/f1b [0,4194304] 0 2024-03-11T19:55:50.194 INFO:tasks.workunit.client.0.smithi107.stdout:3/104: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:50.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/105: write d1/d12/f14 [4757266,56591] 0 2024-03-11T19:55:50.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/55: dwrite f3 [4194304,4194304] 0 2024-03-11T19:55:50.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/69: mkdir d6/d19 0 2024-03-11T19:55:50.213 INFO:tasks.workunit.client.0.smithi107.stdout:8/56: unlink dd/de/lf 0 2024-03-11T19:55:50.213 INFO:tasks.workunit.client.0.smithi107.stdout:8/57: fdatasync f4 0 2024-03-11T19:55:50.219 INFO:tasks.workunit.client.0.smithi107.stdout:1/70: symlink d6/l1a 0 2024-03-11T19:55:50.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/67: dwrite d1/fc [4194304,4194304] 0 2024-03-11T19:55:50.221 INFO:tasks.workunit.client.0.smithi107.stdout:0/80: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:50.221 INFO:tasks.workunit.client.0.smithi107.stdout:1/71: rename d6/f12 to d6/f1b 0 2024-03-11T19:55:50.221 INFO:tasks.workunit.client.0.smithi107.stdout:1/72: chown d6/fd 19689120 1 2024-03-11T19:55:50.221 INFO:tasks.workunit.client.0.smithi107.stdout:1/73: fsync d6/f10 0 2024-03-11T19:55:50.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/81: mknod de/c1d 0 2024-03-11T19:55:50.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/74: mknod d6/c1c 0 2024-03-11T19:55:50.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/75: chown d6/f10 394553356 1 2024-03-11T19:55:50.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/76: truncate d6/f14 19099 0 2024-03-11T19:55:50.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/77: readlink d6/le 0 2024-03-11T19:55:50.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/106: dwrite d1/d12/d1a/f20 [0,4194304] 0 2024-03-11T19:55:50.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/82: dread fd [0,4194304] 0 2024-03-11T19:55:50.255 INFO:tasks.workunit.client.0.smithi107.stdout:0/83: creat de/f1e x:0 0 0 2024-03-11T19:55:50.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/58: dwrite f4 [4194304,4194304] 0 2024-03-11T19:55:50.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/84: rename de/cf to de/c1f 0 2024-03-11T19:55:50.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/85: truncate de/d11/f1a 597937 0 2024-03-11T19:55:50.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/86: readlink lb 0 2024-03-11T19:55:50.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/87: fsync f3 0 2024-03-11T19:55:50.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/59: stat cc 0 2024-03-11T19:55:50.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/60: symlink dd/l10 0 2024-03-11T19:55:50.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/61: chown cb 1 1 2024-03-11T19:55:50.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/62: truncate f3 2801635 0 2024-03-11T19:55:50.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/78: write f4 [554717,8872] 0 2024-03-11T19:55:50.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/68: dwrite d1/fc [4194304,4194304] 0 2024-03-11T19:55:50.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/69: symlink d1/l16 0 2024-03-11T19:55:50.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/79: creat d6/f1d x:0 0 0 2024-03-11T19:55:50.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/80: mkdir d6/d19/d1e 0 2024-03-11T19:55:50.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/81: symlink d6/d19/d1e/l1f 0 2024-03-11T19:55:50.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/82: symlink d6/d19/d1e/l20 0 2024-03-11T19:55:50.288 INFO:tasks.workunit.client.0.smithi107.stdout:0/88: write f5 [129334,17373] 0 2024-03-11T19:55:50.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/89: chown de/d11/c14 380340489 1 2024-03-11T19:55:50.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/107: dread d1/d12/f14 [0,4194304] 0 2024-03-11T19:55:50.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/90: symlink de/l20 0 2024-03-11T19:55:50.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/70: read d1/f6 [3744952,126475] 0 2024-03-11T19:55:50.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/97: sync 2024-03-11T19:55:50.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/98: creat d4/d7/f1d x:0 0 0 2024-03-11T19:55:50.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/99: chown d4/l11 15 1 2024-03-11T19:55:50.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/100: truncate d4/fc 697355 0 2024-03-11T19:55:50.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/83: write d6/f1b [3168017,5334] 0 2024-03-11T19:55:50.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/101: link d4/c13 d4/d7/de/d19/c1e 0 2024-03-11T19:55:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/91: dread de/d11/f1a [0,4194304] 0 2024-03-11T19:55:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/92: truncate f3 4717165 0 2024-03-11T19:55:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/93: write de/f1e [102116,77996] 0 2024-03-11T19:55:50.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/63: dwrite f3 [0,4194304] 0 2024-03-11T19:55:50.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/108: dread d1/fa [0,4194304] 0 2024-03-11T19:55:50.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/102: mkdir d4/d7/d1f 0 2024-03-11T19:55:50.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/109: mkdir d1/d10/d21 0 2024-03-11T19:55:50.312 INFO:tasks.workunit.client.0.smithi107.stdout:3/110: symlink d1/d12/l22 0 2024-03-11T19:55:50.314 INFO:tasks.workunit.client.0.smithi107.stdout:3/111: mknod d1/d19/c23 0 2024-03-11T19:55:50.316 INFO:tasks.workunit.client.0.smithi107.stdout:3/112: rename d1/d12/l22 to d1/d19/l24 0 2024-03-11T19:55:50.316 INFO:tasks.workunit.client.0.smithi107.stdout:3/113: truncate d1/d10/d13/d16/d17/f1f 202715 0 2024-03-11T19:55:50.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/114: read d1/d10/d13/d16/d17/f1f [55353,99323] 0 2024-03-11T19:55:50.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/103: dread f0 [0,4194304] 0 2024-03-11T19:55:50.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/104: write d4/fc [5123,101953] 0 2024-03-11T19:55:50.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/105: fdatasync f3 0 2024-03-11T19:55:50.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/106: unlink d4/l5 0 2024-03-11T19:55:50.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/107: stat d4/d7 0 2024-03-11T19:55:50.334 INFO:tasks.workunit.client.0.smithi107.stdout:4/108: creat d4/d7/d1f/f20 x:0 0 0 2024-03-11T19:55:50.334 INFO:tasks.workunit.client.0.smithi107.stdout:8/64: dread f3 [0,4194304] 0 2024-03-11T19:55:50.337 INFO:tasks.workunit.client.0.smithi107.stdout:4/109: creat d4/d7/db/f21 x:0 0 0 2024-03-11T19:55:50.338 INFO:tasks.workunit.client.0.smithi107.stdout:8/65: link c2 dd/de/c11 0 2024-03-11T19:55:50.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/94: dwrite de/f16 [4194304,4194304] 0 2024-03-11T19:55:50.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/95: truncate de/d11/f1b 1944941 0 2024-03-11T19:55:50.348 INFO:tasks.workunit.client.0.smithi107.stdout:0/96: creat de/f21 x:0 0 0 2024-03-11T19:55:50.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/97: dread f5 [0,4194304] 0 2024-03-11T19:55:50.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/98: mknod de/d11/c22 0 2024-03-11T19:55:50.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/99: mkdir de/d11/d23 0 2024-03-11T19:55:50.370 INFO:tasks.workunit.client.0.smithi107.stdout:0/100: mkdir de/d11/d23/d24 0 2024-03-11T19:55:50.370 INFO:tasks.workunit.client.0.smithi107.stdout:0/101: chown l2 7 1 2024-03-11T19:55:50.371 INFO:tasks.workunit.client.0.smithi107.stdout:0/102: truncate f5 2416997 0 2024-03-11T19:55:50.373 INFO:tasks.workunit.client.0.smithi107.stdout:0/103: unlink de/l12 0 2024-03-11T19:55:50.373 INFO:tasks.workunit.client.0.smithi107.stdout:0/104: fsync f3 0 2024-03-11T19:55:50.374 INFO:tasks.workunit.client.0.smithi107.stdout:1/84: dwrite d6/f1d [0,4194304] 0 2024-03-11T19:55:50.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/105: read de/d11/f1a [135311,45867] 0 2024-03-11T19:55:50.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/85: rename d6/fc to d6/d19/d1e/f21 0 2024-03-11T19:55:50.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/86: readlink l1 0 2024-03-11T19:55:50.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/87: truncate d6/f10 76381 0 2024-03-11T19:55:50.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/88: mknod d6/d19/c22 0 2024-03-11T19:55:50.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/89: read d6/f10 [53801,97106] 0 2024-03-11T19:55:50.380 INFO:tasks.workunit.client.0.smithi107.stdout:5/106: sync 2024-03-11T19:55:50.380 INFO:tasks.workunit.client.0.smithi107.stdout:5/107: read - da/d14/d19/f1d zero size 2024-03-11T19:55:50.380 INFO:tasks.workunit.client.0.smithi107.stdout:5/108: dread - da/d14/d19/f1d zero size 2024-03-11T19:55:50.385 INFO:tasks.workunit.client.0.smithi107.stdout:5/109: symlink da/d14/l22 0 2024-03-11T19:55:50.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/94: sync 2024-03-11T19:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout:4/110: dwrite d4/fc [0,4194304] 0 2024-03-11T19:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/111: chown d4/d7/db/f21 314190 1 2024-03-11T19:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/112: creat d4/d7/f22 x:0 0 0 2024-03-11T19:55:50.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/95: dread d6/d9/d13/f14 [0,4194304] 0 2024-03-11T19:55:50.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/96: write d6/fb [4262377,69386] 0 2024-03-11T19:55:50.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/97: readlink - no filename 2024-03-11T19:55:50.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/90: dread d6/f15 [0,4194304] 0 2024-03-11T19:55:50.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/91: readlink d6/le 0 2024-03-11T19:55:50.401 INFO:tasks.workunit.client.0.smithi107.stdout:1/92: fdatasync d6/f1d 0 2024-03-11T19:55:50.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/98: creat d6/d9/f19 x:0 0 0 2024-03-11T19:55:50.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/99: rename d6/d9/f19 to d6/da/d16/f1a 0 2024-03-11T19:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout:0/106: dwrite fd [0,4194304] 0 2024-03-11T19:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/107: write de/d11/f1a [1500625,10478] 0 2024-03-11T19:55:50.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/100: write d6/fb [3356085,123571] 0 2024-03-11T19:55:50.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/110: dwrite f5 [4194304,4194304] 0 2024-03-11T19:55:50.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/111: truncate f9 5647391 0 2024-03-11T19:55:50.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/112: creat da/dd/f23 x:0 0 0 2024-03-11T19:55:50.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/113: stat da/fb 0 2024-03-11T19:55:50.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/113: dwrite d4/d7/de/f15 [0,4194304] 0 2024-03-11T19:55:50.440 INFO:tasks.workunit.client.0.smithi107.stdout:0/108: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:50.440 INFO:tasks.workunit.client.0.smithi107.stdout:0/109: readlink de/d11/l19 0 2024-03-11T19:55:50.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/114: getdents d4/d7/de 0 2024-03-11T19:55:50.444 INFO:tasks.workunit.client.0.smithi107.stdout:4/115: creat d4/d7/de/d19/f23 x:0 0 0 2024-03-11T19:55:50.446 INFO:tasks.workunit.client.0.smithi107.stdout:0/110: dread de/f13 [0,4194304] 0 2024-03-11T19:55:50.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/111: write fd [8390264,41331] 0 2024-03-11T19:55:50.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/116: mkdir d4/d7/de/d24 0 2024-03-11T19:55:50.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/93: dwrite d6/f10 [0,4194304] 0 2024-03-11T19:55:50.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/117: symlink d4/d7/d1f/l25 0 2024-03-11T19:55:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:7/101: dread d6/fe [0,4194304] 0 2024-03-11T19:55:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:7/102: write d6/da/d16/f1a [527037,125096] 0 2024-03-11T19:55:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:1/94: getdents d6/d19 0 2024-03-11T19:55:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/112: symlink de/d11/l25 0 2024-03-11T19:55:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/114: dread f5 [4194304,4194304] 0 2024-03-11T19:55:50.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/118: mkdir d4/d7/de/d24/d26 0 2024-03-11T19:55:50.467 INFO:tasks.workunit.client.0.smithi107.stdout:7/103: mkdir d6/d9/d13/d1b 0 2024-03-11T19:55:50.467 INFO:tasks.workunit.client.0.smithi107.stdout:7/104: truncate d6/d9/d13/f14 926742 0 2024-03-11T19:55:50.467 INFO:tasks.workunit.client.0.smithi107.stdout:7/105: write d6/fe [3874134,73594] 0 2024-03-11T19:55:50.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/115: mknod da/d14/c24 0 2024-03-11T19:55:50.468 INFO:tasks.workunit.client.0.smithi107.stdout:1/95: symlink d6/l23 0 2024-03-11T19:55:50.470 INFO:tasks.workunit.client.0.smithi107.stdout:1/96: creat d6/d19/d1e/f24 x:0 0 0 2024-03-11T19:55:50.470 INFO:tasks.workunit.client.0.smithi107.stdout:7/106: rename d6/fb to d6/d9/d13/d1b/f1c 0 2024-03-11T19:55:50.470 INFO:tasks.workunit.client.0.smithi107.stdout:1/97: symlink d6/d19/l25 0 2024-03-11T19:55:50.471 INFO:tasks.workunit.client.0.smithi107.stdout:1/98: chown d6/fd 42 1 2024-03-11T19:55:50.471 INFO:tasks.workunit.client.0.smithi107.stdout:1/99: readlink d6/l13 0 2024-03-11T19:55:50.475 INFO:tasks.workunit.client.0.smithi107.stdout:6/93: sync 2024-03-11T19:55:50.475 INFO:tasks.workunit.client.0.smithi107.stdout:9/79: sync 2024-03-11T19:55:50.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/71: sync 2024-03-11T19:55:50.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/72: truncate d1/f15 802811 0 2024-03-11T19:55:50.480 INFO:tasks.workunit.client.0.smithi107.stdout:2/73: link d1/l2 d1/l17 0 2024-03-11T19:55:50.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/74: unlink d1/cf 0 2024-03-11T19:55:50.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/100: stat d6 0 2024-03-11T19:55:50.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/101: stat d6/f14 0 2024-03-11T19:55:50.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/102: write d6/fd [296318,74430] 0 2024-03-11T19:55:50.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/107: dread d6/fe [0,4194304] 0 2024-03-11T19:55:50.507 INFO:tasks.workunit.client.0.smithi107.stdout:7/108: write d6/da/d16/f1a [1034773,41368] 0 2024-03-11T19:55:50.507 INFO:tasks.workunit.client.0.smithi107.stdout:7/109: write d6/d9/d13/d1b/f1c [4753485,63403] 0 2024-03-11T19:55:50.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/94: dread fc [0,4194304] 0 2024-03-11T19:55:50.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/95: creat d10/d16/f1c x:0 0 0 2024-03-11T19:55:50.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/116: dwrite da/dd/f21 [0,4194304] 0 2024-03-11T19:55:50.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/117: rmdir da/d14 39 2024-03-11T19:55:50.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/118: truncate da/dd/f1f 153285 0 2024-03-11T19:55:50.540 INFO:tasks.workunit.client.0.smithi107.stdout:4/119: dwrite d4/d7/de/f15 [0,4194304] 0 2024-03-11T19:55:50.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/96: dwrite f8 [0,4194304] 0 2024-03-11T19:55:50.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/97: link lb d10/d16/l1d 0 2024-03-11T19:55:50.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/98: write f0 [5472880,62588] 0 2024-03-11T19:55:50.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/99: fdatasync f3 0 2024-03-11T19:55:50.551 INFO:tasks.workunit.client.0.smithi107.stdout:6/100: creat d10/d16/f1e x:0 0 0 2024-03-11T19:55:50.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/75: dread d1/fc [4194304,4194304] 0 2024-03-11T19:55:50.552 INFO:tasks.workunit.client.0.smithi107.stdout:6/101: mkdir d10/d16/d1f 0 2024-03-11T19:55:50.552 INFO:tasks.workunit.client.0.smithi107.stdout:6/102: creat d10/d16/f20 x:0 0 0 2024-03-11T19:55:50.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/76: write d1/f10 [1388505,37545] 0 2024-03-11T19:55:50.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/77: fsync d1/f6 0 2024-03-11T19:55:50.592 INFO:tasks.workunit.client.0.smithi107.stdout:4/120: dwrite d4/d7/de/d19/f23 [0,4194304] 0 2024-03-11T19:55:50.596 INFO:tasks.workunit.client.0.smithi107.stdout:9/80: dwrite f1 [0,4194304] 0 2024-03-11T19:55:50.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/119: dwrite da/f1a [0,4194304] 0 2024-03-11T19:55:50.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/110: read d6/da/d16/f1a [991043,116086] 0 2024-03-11T19:55:50.596 INFO:tasks.workunit.client.0.smithi107.stdout:4/121: mkdir d4/d7/de/d19/d27 0 2024-03-11T19:55:50.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/81: mkdir d4/d1d 0 2024-03-11T19:55:50.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/82: fsync d4/d9/f15 0 2024-03-11T19:55:50.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/83: dread - d4/d10/d17/f1a zero size 2024-03-11T19:55:50.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/84: write d4/d10/d17/f1b [21694,90984] 0 2024-03-11T19:55:50.600 INFO:tasks.workunit.client.0.smithi107.stdout:6/103: dwrite f0 [0,4194304] 0 2024-03-11T19:55:50.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/111: mknod d6/d9/c1d 0 2024-03-11T19:55:50.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/112: stat d6/d9/d13/c12 0 2024-03-11T19:55:50.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/113: fsync d6/fe 0 2024-03-11T19:55:50.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/78: dwrite d1/f15 [0,4194304] 0 2024-03-11T19:55:50.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/122: link d4/d7/de/d19/l1c d4/d7/de/d24/d26/l28 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/85: link d4/d9/f15 d4/d1d/f1e 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/123: truncate d4/d7/de/d19/f23 4594643 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/86: dread - d4/fe zero size 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/104: creat d10/d16/d1f/f21 x:0 0 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/114: unlink d6/cc 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/105: truncate d10/d16/f20 160718 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/106: rename d10/d16 to d10/d16/d22 22 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/115: write d6/d9/d13/f14 [92310,1106] 0 2024-03-11T19:55:50.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/107: read - d10/d16/f1c zero size 2024-03-11T19:55:50.616 INFO:tasks.workunit.client.0.smithi107.stdout:9/87: mkdir d4/d1d/d1f 0 2024-03-11T19:55:50.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/88: fsync d4/fe 0 2024-03-11T19:55:50.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/89: fsync d4/d10/d17/f1a 0 2024-03-11T19:55:50.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/90: stat d4/d1d 0 2024-03-11T19:55:50.617 INFO:tasks.workunit.client.0.smithi107.stdout:5/120: read da/dd/f1f [2850662,94807] 0 2024-03-11T19:55:50.617 INFO:tasks.workunit.client.0.smithi107.stdout:5/121: write da/dd/f13 [1264289,95480] 0 2024-03-11T19:55:50.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/116: unlink d6/d9/d13/f14 0 2024-03-11T19:55:50.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/117: write d6/d9/f8 [253285,52890] 0 2024-03-11T19:55:50.620 INFO:tasks.workunit.client.0.smithi107.stdout:6/108: creat d10/d19/f23 x:0 0 0 2024-03-11T19:55:50.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/122: read da/dd/f13 [551704,40758] 0 2024-03-11T19:55:50.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/118: rmdir d6/d9/d13/d1b 39 2024-03-11T19:55:50.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/109: mkdir d10/d16/d24 0 2024-03-11T19:55:50.626 INFO:tasks.workunit.client.0.smithi107.stdout:5/123: mknod da/d14/c25 0 2024-03-11T19:55:50.627 INFO:tasks.workunit.client.0.smithi107.stdout:3/115: sync 2024-03-11T19:55:50.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/66: sync 2024-03-11T19:55:50.627 INFO:tasks.workunit.client.0.smithi107.stdout:5/124: write da/dd/f13 [837808,70713] 0 2024-03-11T19:55:50.627 INFO:tasks.workunit.client.0.smithi107.stdout:5/125: write f9 [4415174,115675] 0 2024-03-11T19:55:50.627 INFO:tasks.workunit.client.0.smithi107.stdout:5/126: dread - da/dd/f23 zero size 2024-03-11T19:55:50.628 INFO:tasks.workunit.client.0.smithi107.stdout:3/116: creat d1/d10/d13/d16/f25 x:0 0 0 2024-03-11T19:55:50.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/67: unlink dd/l10 0 2024-03-11T19:55:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/127: rename da/dd/ce to da/dd/c26 0 2024-03-11T19:55:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/128: chown da/d14/d19/f1d 393354219 1 2024-03-11T19:55:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/129: chown da/d14/c24 29 1 2024-03-11T19:55:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/79: write d1/fc [3294661,119531] 0 2024-03-11T19:55:50.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/80: chown d1/l5 2713 1 2024-03-11T19:55:50.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/68: link l5 dd/l12 0 2024-03-11T19:55:50.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/119: dread d6/fe [4194304,4194304] 0 2024-03-11T19:55:50.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/120: chown d6 9 1 2024-03-11T19:55:50.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/121: write d6/fe [3811027,65075] 0 2024-03-11T19:55:50.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/130: creat da/d14/d18/f27 x:0 0 0 2024-03-11T19:55:50.638 INFO:tasks.workunit.client.0.smithi107.stdout:7/122: link d6/f17 d6/da/f1e 0 2024-03-11T19:55:50.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/123: creat d6/da/f1f x:0 0 0 2024-03-11T19:55:50.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/124: write d6/f17 [5379166,28562] 0 2024-03-11T19:55:50.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/131: rename da/dd/f21 to da/dd/f28 0 2024-03-11T19:55:50.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/132: read - da/d14/d19/f1d zero size 2024-03-11T19:55:50.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/133: dread - da/d14/d18/f1c zero size 2024-03-11T19:55:50.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/125: symlink d6/d9/d13/d1b/l20 0 2024-03-11T19:55:50.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/81: truncate d1/f15 1082076 0 2024-03-11T19:55:50.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/126: unlink d6/fe 0 2024-03-11T19:55:50.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/127: write d6/da/f1f [560031,41985] 0 2024-03-11T19:55:50.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/110: dread f3 [0,4194304] 0 2024-03-11T19:55:50.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/111: link d10/f1a d10/d16/d1f/f25 0 2024-03-11T19:55:50.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/124: dwrite d4/d7/de/f15 [4194304,4194304] 0 2024-03-11T19:55:50.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/125: chown d4/d7/de/l16 0 1 2024-03-11T19:55:50.658 INFO:tasks.workunit.client.0.smithi107.stdout:4/126: chown d4/d7/de/d24/d26/l28 9616401 1 2024-03-11T19:55:50.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/112: mknod d10/d16/c26 0 2024-03-11T19:55:50.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/113: write d10/f1a [1099845,73493] 0 2024-03-11T19:55:50.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/114: stat d10/d19 0 2024-03-11T19:55:50.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/115: chown d10/d16/d1f 140716162 1 2024-03-11T19:55:50.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/69: write f3 [1836502,109012] 0 2024-03-11T19:55:50.677 INFO:tasks.workunit.client.0.smithi107.stdout:3/117: dwrite d1/d10/d13/d16/d17/f1f [0,4194304] 0 2024-03-11T19:55:50.678 INFO:tasks.workunit.client.0.smithi107.stdout:3/118: fdatasync d1/d10/d13/d16/f25 0 2024-03-11T19:55:50.678 INFO:tasks.workunit.client.0.smithi107.stdout:2/82: dread d1/f6 [0,4194304] 0 2024-03-11T19:55:50.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/83: chown d1/l5 198897094 1 2024-03-11T19:55:50.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/84: creat d1/f18 x:0 0 0 2024-03-11T19:55:50.680 INFO:tasks.workunit.client.0.smithi107.stdout:2/85: truncate d1/f10 4179427 0 2024-03-11T19:55:50.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/86: mknod d1/c19 0 2024-03-11T19:55:50.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/87: fdatasync d1/fc 0 2024-03-11T19:55:50.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/88: mkdir d1/d1a 0 2024-03-11T19:55:50.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/89: write d1/f10 [843989,37437] 0 2024-03-11T19:55:50.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/90: mknod d1/d1a/c1b 0 2024-03-11T19:55:50.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/91: dread - d1/f18 zero size 2024-03-11T19:55:50.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/92: truncate d1/f6 2156696 0 2024-03-11T19:55:50.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/127: write d4/d7/de/d19/f23 [858743,51875] 0 2024-03-11T19:55:50.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/128: mkdir d4/d7/de/d24/d26/d29 0 2024-03-11T19:55:50.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/129: fdatasync d4/d7/d1f/f20 0 2024-03-11T19:55:50.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/128: dread d6/da/f1e [4194304,4194304] 0 2024-03-11T19:55:50.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/129: write d6/da/d16/f1a [1367930,20350] 0 2024-03-11T19:55:50.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/130: mkdir d6/d21 0 2024-03-11T19:55:50.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/131: creat d6/f22 x:0 0 0 2024-03-11T19:55:50.693 INFO:tasks.workunit.client.0.smithi107.stdout:7/132: getdents d6/da/d16 0 2024-03-11T19:55:50.694 INFO:tasks.workunit.client.0.smithi107.stdout:8/70: dread f3 [0,4194304] 0 2024-03-11T19:55:50.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/116: dread d10/f1a [0,4194304] 0 2024-03-11T19:55:50.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/117: write d10/d16/f1c [963665,3870] 0 2024-03-11T19:55:50.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/118: readlink d10/l11 0 2024-03-11T19:55:50.695 INFO:tasks.workunit.client.0.smithi107.stdout:7/133: link d6/d9/d13/d1b/f1c d6/d9/f23 0 2024-03-11T19:55:50.695 INFO:tasks.workunit.client.0.smithi107.stdout:7/134: chown d6/da/d16/c18 1603607 1 2024-03-11T19:55:50.696 INFO:tasks.workunit.client.0.smithi107.stdout:7/135: creat d6/da/f24 x:0 0 0 2024-03-11T19:55:50.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/71: mkdir dd/d13 0 2024-03-11T19:55:50.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/72: readlink l9 0 2024-03-11T19:55:50.707 INFO:tasks.workunit.client.0.smithi107.stdout:7/136: dread d6/d9/f23 [4194304,4194304] 0 2024-03-11T19:55:50.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/73: truncate f3 125371 0 2024-03-11T19:55:50.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/74: creat dd/f14 x:0 0 0 2024-03-11T19:55:50.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/75: creat dd/de/f15 x:0 0 0 2024-03-11T19:55:50.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/76: mkdir dd/d16 0 2024-03-11T19:55:50.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/77: fdatasync f4 0 2024-03-11T19:55:50.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/78: getdents dd 0 2024-03-11T19:55:50.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/79: rename c2 to dd/c17 0 2024-03-11T19:55:50.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/80: readlink dd/l12 0 2024-03-11T19:55:50.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/81: stat dd/de/c11 0 2024-03-11T19:55:50.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/82: mkdir dd/d13/d18 0 2024-03-11T19:55:50.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/83: fsync f4 0 2024-03-11T19:55:50.722 INFO:tasks.workunit.client.0.smithi107.stdout:8/84: truncate f3 1149289 0 2024-03-11T19:55:50.722 INFO:tasks.workunit.client.0.smithi107.stdout:8/85: write f3 [10314,51084] 0 2024-03-11T19:55:50.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/86: unlink cb 0 2024-03-11T19:55:50.726 INFO:tasks.workunit.client.0.smithi107.stdout:8/87: creat dd/d16/f19 x:0 0 0 2024-03-11T19:55:50.727 INFO:tasks.workunit.client.0.smithi107.stdout:8/88: dread - dd/f14 zero size 2024-03-11T19:55:50.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/89: read - dd/de/f15 zero size 2024-03-11T19:55:50.731 INFO:tasks.workunit.client.0.smithi107.stdout:8/90: unlink dd/f14 0 2024-03-11T19:55:50.731 INFO:tasks.workunit.client.0.smithi107.stdout:8/91: write dd/de/f15 [709838,94945] 0 2024-03-11T19:55:50.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/92: link dd/d16/f19 dd/d13/d18/f1a 0 2024-03-11T19:55:50.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/93: write dd/de/f15 [1747305,76539] 0 2024-03-11T19:55:50.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/94: fdatasync f4 0 2024-03-11T19:55:50.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/95: dread - dd/d16/f19 zero size 2024-03-11T19:55:50.737 INFO:tasks.workunit.client.0.smithi107.stdout:3/119: dwrite d1/d10/d13/d16/f25 [0,4194304] 0 2024-03-11T19:55:50.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/130: dwrite d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:50.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/119: dwrite d10/d19/f23 [0,4194304] 0 2024-03-11T19:55:50.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/120: write d10/d16/d1f/f25 [1470317,120154] 0 2024-03-11T19:55:50.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/120: unlink c0 0 2024-03-11T19:55:50.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/96: mkdir dd/de/d1b 0 2024-03-11T19:55:50.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/97: truncate dd/d16/f19 868074 0 2024-03-11T19:55:50.746 INFO:tasks.workunit.client.0.smithi107.stdout:6/121: creat d10/d16/d24/f27 x:0 0 0 2024-03-11T19:55:50.747 INFO:tasks.workunit.client.0.smithi107.stdout:3/121: truncate d1/fa 824655 0 2024-03-11T19:55:50.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/122: rename d10/d16/d1f/f21 to d10/d16/d1f/f28 0 2024-03-11T19:55:50.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/123: fdatasync f0 0 2024-03-11T19:55:50.750 INFO:tasks.workunit.client.0.smithi107.stdout:8/98: truncate f3 754812 0 2024-03-11T19:55:50.753 INFO:tasks.workunit.client.0.smithi107.stdout:6/124: mkdir d10/d16/d1f/d29 0 2024-03-11T19:55:50.753 INFO:tasks.workunit.client.0.smithi107.stdout:4/131: write f0 [1851170,67046] 0 2024-03-11T19:55:50.753 INFO:tasks.workunit.client.0.smithi107.stdout:4/132: creat d4/d7/de/d19/f2a x:0 0 0 2024-03-11T19:55:50.753 INFO:tasks.workunit.client.0.smithi107.stdout:4/133: read f3 [1316093,115504] 0 2024-03-11T19:55:50.754 INFO:tasks.workunit.client.0.smithi107.stdout:8/99: creat dd/d13/d18/f1c x:0 0 0 2024-03-11T19:55:50.754 INFO:tasks.workunit.client.0.smithi107.stdout:3/122: fsync d1/fa 0 2024-03-11T19:55:50.755 INFO:tasks.workunit.client.0.smithi107.stdout:7/137: write d6/ff [851270,95903] 0 2024-03-11T19:55:50.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/93: dwrite d1/fc [8388608,4194304] 0 2024-03-11T19:55:50.764 INFO:tasks.workunit.client.0.smithi107.stdout:7/138: mknod d6/d9/d13/d1b/c25 0 2024-03-11T19:55:50.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/139: read - d6/f22 zero size 2024-03-11T19:55:50.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/134: rename d4/d7/de to d4/d7/d1f/d2b 0 2024-03-11T19:55:50.770 INFO:tasks.workunit.client.0.smithi107.stdout:7/140: creat d6/d21/f26 x:0 0 0 2024-03-11T19:55:50.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/141: read d6/d9/d13/d1b/f1c [5350760,74311] 0 2024-03-11T19:55:50.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/135: getdents d4/d7 0 2024-03-11T19:55:50.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/94: dread d1/f10 [0,4194304] 0 2024-03-11T19:55:50.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/142: unlink d6/d9/f11 0 2024-03-11T19:55:50.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/143: chown d6/d9/d13 1810680 1 2024-03-11T19:55:50.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/123: rename d1/d10/d13/d16 to d1/d10/d26 0 2024-03-11T19:55:50.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/124: creat d1/d10/d21/f27 x:0 0 0 2024-03-11T19:55:50.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/125: truncate d1/d12/d1a/f20 2329788 0 2024-03-11T19:55:50.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/126: creat d1/d10/d21/f28 x:0 0 0 2024-03-11T19:55:50.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/127: read d1/f5 [3948569,113628] 0 2024-03-11T19:55:50.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/128: mknod d1/d10/c29 0 2024-03-11T19:55:50.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/136: dread d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:50.804 INFO:tasks.workunit.client.0.smithi107.stdout:3/129: mkdir d1/d10/d2a 0 2024-03-11T19:55:50.804 INFO:tasks.workunit.client.0.smithi107.stdout:3/130: readlink d1/l4 0 2024-03-11T19:55:50.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/125: dwrite d10/f1a [0,4194304] 0 2024-03-11T19:55:50.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/137: creat d4/d7/f2c x:0 0 0 2024-03-11T19:55:50.806 INFO:tasks.workunit.client.0.smithi107.stdout:6/126: unlink d10/f1a 0 2024-03-11T19:55:50.806 INFO:tasks.workunit.client.0.smithi107.stdout:6/127: chown d10/d16/f17 413492431 1 2024-03-11T19:55:50.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/138: creat d4/d7/d1f/d2b/d24/d26/d29/f2d x:0 0 0 2024-03-11T19:55:50.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/139: dread - d4/d7/f2c zero size 2024-03-11T19:55:50.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/140: truncate d4/d7/f2c 962915 0 2024-03-11T19:55:50.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/128: link d10/d16/f17 d10/d16/d1f/f2a 0 2024-03-11T19:55:50.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/129: chown d10/d16/d1f/f25 2 1 2024-03-11T19:55:50.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/130: readlink d10/d19/l1b 0 2024-03-11T19:55:50.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/141: unlink d4/c1a 0 2024-03-11T19:55:50.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/144: dread d6/f17 [0,4194304] 0 2024-03-11T19:55:50.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/142: mkdir d4/d7/d1f/d2b/d24/d26/d29/d2e 0 2024-03-11T19:55:50.812 INFO:tasks.workunit.client.0.smithi107.stdout:7/145: rename d6/d9/d13/d1b/l20 to d6/da/l27 0 2024-03-11T19:55:50.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/146: truncate d6/ff 521107 0 2024-03-11T19:55:50.832 INFO:tasks.workunit.client.0.smithi107.stdout:2/95: dwrite d1/f10 [4194304,4194304] 0 2024-03-11T19:55:50.832 INFO:tasks.workunit.client.0.smithi107.stdout:2/96: readlink d1/l9 0 2024-03-11T19:55:50.832 INFO:tasks.workunit.client.0.smithi107.stdout:2/97: stat d1/l9 0 2024-03-11T19:55:50.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/98: symlink d1/l1c 0 2024-03-11T19:55:50.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/99: chown d1/c13 1407692346 1 2024-03-11T19:55:50.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/131: dread d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:50.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/132: rename d1/d19/c23 to d1/d10/d21/c2b 0 2024-03-11T19:55:50.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/133: creat d1/d10/d13/f2c x:0 0 0 2024-03-11T19:55:50.862 INFO:tasks.workunit.client.0.smithi107.stdout:7/147: dwrite d6/f22 [0,4194304] 0 2024-03-11T19:55:50.862 INFO:tasks.workunit.client.0.smithi107.stdout:7/148: fsync d6/f17 0 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:4/143: dread d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:4/144: read - d4/d7/db/f21 zero size 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:6/131: dwrite d10/d16/f1c [0,4194304] 0 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:6/132: fsync d10/d16/d1f/f28 0 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/149: mknod d6/d9/d13/c28 0 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/150: dread - d6/da/f24 zero size 2024-03-11T19:55:50.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/151: creat d6/f29 x:0 0 0 2024-03-11T19:55:50.870 INFO:tasks.workunit.client.0.smithi107.stdout:6/133: mkdir d10/d16/d2b 0 2024-03-11T19:55:50.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/103: sync 2024-03-11T19:55:50.880 INFO:tasks.workunit.client.0.smithi107.stdout:3/134: read d1/d12/f14 [39922,9101] 0 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/152: dread d6/da/d16/f1a [0,4194304] 0 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/153: chown d6/d9/d13 1352 1 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/154: creat d6/d21/f2a x:0 0 0 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/155: write d6/d21/f26 [624970,129216] 0 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/156: readlink d6/da/l27 0 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/157: dread - d6/da/f24 zero size 2024-03-11T19:55:50.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/158: write d6/d9/f23 [935972,32553] 0 2024-03-11T19:55:50.883 INFO:tasks.workunit.client.0.smithi107.stdout:7/159: creat d6/da/d16/f2b x:0 0 0 2024-03-11T19:55:50.884 INFO:tasks.workunit.client.0.smithi107.stdout:2/100: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:50.888 INFO:tasks.workunit.client.0.smithi107.stdout:7/160: rename d6/da/f24 to d6/da/d16/f2c 0 2024-03-11T19:55:50.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/135: dread d1/f5 [4194304,4194304] 0 2024-03-11T19:55:50.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/101: mkdir d1/d1d 0 2024-03-11T19:55:50.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/102: getdents d1/d1d 0 2024-03-11T19:55:50.890 INFO:tasks.workunit.client.0.smithi107.stdout:3/136: mkdir d1/d19/d2d 0 2024-03-11T19:55:50.896 INFO:tasks.workunit.client.0.smithi107.stdout:7/161: dread d6/da/f1f [0,4194304] 0 2024-03-11T19:55:50.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/103: mkdir d1/d1a/d1e 0 2024-03-11T19:55:50.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/104: chown d1/f15 29 1 2024-03-11T19:55:50.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/105: chown d1/f18 22 1 2024-03-11T19:55:50.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/106: fdatasync d1/fc 0 2024-03-11T19:55:50.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/100: sync 2024-03-11T19:55:50.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/113: sync 2024-03-11T19:55:50.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/91: sync 2024-03-11T19:55:50.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/134: sync 2024-03-11T19:55:50.899 INFO:tasks.workunit.client.0.smithi107.stdout:0/114: readlink l1 0 2024-03-11T19:55:50.899 INFO:tasks.workunit.client.0.smithi107.stdout:1/104: dread d6/f15 [0,4194304] 0 2024-03-11T19:55:50.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/107: rename d1/c12 to d1/d1d/c1f 0 2024-03-11T19:55:50.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/135: mknod da/d14/d19/c29 0 2024-03-11T19:55:50.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/136: fdatasync da/dd/f13 0 2024-03-11T19:55:50.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/137: fsync da/dd/f11 0 2024-03-11T19:55:50.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/138: chown da/dd/f11 2425 1 2024-03-11T19:55:50.901 INFO:tasks.workunit.client.0.smithi107.stdout:9/92: symlink d4/d1d/d1f/l20 0 2024-03-11T19:55:50.901 INFO:tasks.workunit.client.0.smithi107.stdout:9/93: fsync d4/f12 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/94: readlink d4/ld 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:8/101: write dd/de/f15 [938062,80458] 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/108: creat d1/d1a/f20 x:0 0 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:5/139: rmdir da/d14/d19 39 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:5/140: stat da/dd/f28 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/95: mknod d4/d1d/d1f/c21 0 2024-03-11T19:55:50.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/96: write d4/d9/f15 [1275868,116941] 0 2024-03-11T19:55:50.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/97: write d4/d1d/f1e [712764,28342] 0 2024-03-11T19:55:50.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/102: creat dd/de/d1b/f1d x:0 0 0 2024-03-11T19:55:50.904 INFO:tasks.workunit.client.0.smithi107.stdout:2/109: mkdir d1/d21 0 2024-03-11T19:55:50.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/110: chown d1/f10 93007 1 2024-03-11T19:55:50.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/111: creat d1/f22 x:0 0 0 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:8/103: unlink cc 0 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:8/104: chown dd/d13/d18/f1a 154809 1 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/112: link d1/f10 d1/d1a/f23 0 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/113: dread - d1/f18 zero size 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/114: dread - d1/d1a/f20 zero size 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/115: readlink d1/l9 0 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/116: getdents d1/d1a/d1e 0 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/117: read - d1/f18 zero size 2024-03-11T19:55:50.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/145: write d4/d7/d1f/f20 [228115,59007] 0 2024-03-11T19:55:50.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/118: creat d1/d1d/f24 x:0 0 0 2024-03-11T19:55:50.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/119: symlink d1/d1a/d1e/l25 0 2024-03-11T19:55:50.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/120: fsync d1/f15 0 2024-03-11T19:55:50.913 INFO:tasks.workunit.client.0.smithi107.stdout:1/105: write d6/fd [263092,72817] 0 2024-03-11T19:55:50.913 INFO:tasks.workunit.client.0.smithi107.stdout:1/106: creat d6/d19/d1e/f26 x:0 0 0 2024-03-11T19:55:50.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/121: mkdir d1/d1a/d1e/d26 0 2024-03-11T19:55:50.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/162: read d6/f22 [2583516,120541] 0 2024-03-11T19:55:50.927 INFO:tasks.workunit.client.0.smithi107.stdout:1/107: mkdir d6/d19/d1e/d27 0 2024-03-11T19:55:50.927 INFO:tasks.workunit.client.0.smithi107.stdout:5/141: write da/dd/f28 [331540,57550] 0 2024-03-11T19:55:50.928 INFO:tasks.workunit.client.0.smithi107.stdout:5/142: write da/dd/f28 [5035093,62847] 0 2024-03-11T19:55:50.928 INFO:tasks.workunit.client.0.smithi107.stdout:5/143: truncate da/dd/f23 678534 0 2024-03-11T19:55:50.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/134: dwrite d10/f14 [0,4194304] 0 2024-03-11T19:55:50.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/115: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:50.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/116: dread - de/f17 zero size 2024-03-11T19:55:50.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/117: fdatasync de/d11/f1a 0 2024-03-11T19:55:50.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/135: write fa [102608,86154] 0 2024-03-11T19:55:50.931 INFO:tasks.workunit.client.0.smithi107.stdout:5/144: truncate f5 7514028 0 2024-03-11T19:55:50.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/118: mkdir de/d11/d23/d26 0 2024-03-11T19:55:50.935 INFO:tasks.workunit.client.0.smithi107.stdout:5/145: dread da/dd/f23 [0,4194304] 0 2024-03-11T19:55:50.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/136: symlink d10/d16/d24/l2c 0 2024-03-11T19:55:50.937 INFO:tasks.workunit.client.0.smithi107.stdout:9/98: dread d4/d10/d17/f19 [0,4194304] 0 2024-03-11T19:55:50.938 INFO:tasks.workunit.client.0.smithi107.stdout:5/146: symlink da/d14/d18/d1e/l2a 0 2024-03-11T19:55:50.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/122: getdents d1 0 2024-03-11T19:55:50.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/123: fdatasync d1/d1d/f24 0 2024-03-11T19:55:50.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/147: link da/d14/c25 da/d14/d19/c2b 0 2024-03-11T19:55:50.941 INFO:tasks.workunit.client.0.smithi107.stdout:5/148: write da/d14/d18/f1c [402007,116473] 0 2024-03-11T19:55:50.941 INFO:tasks.workunit.client.0.smithi107.stdout:5/149: truncate da/d14/d18/f1c 770213 0 2024-03-11T19:55:50.948 INFO:tasks.workunit.client.0.smithi107.stdout:2/124: symlink d1/l27 0 2024-03-11T19:55:50.948 INFO:tasks.workunit.client.0.smithi107.stdout:2/125: readlink d1/l27 0 2024-03-11T19:55:50.948 INFO:tasks.workunit.client.0.smithi107.stdout:2/126: stat d1/l1c 0 2024-03-11T19:55:50.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/127: rename d1/f22 to d1/d21/f28 0 2024-03-11T19:55:50.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/128: readlink d1/l1c 0 2024-03-11T19:55:50.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/129: write d1/d1d/f24 [374811,111862] 0 2024-03-11T19:55:50.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/108: read d6/f10 [2119570,101363] 0 2024-03-11T19:55:50.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/109: dread - d6/f16 zero size 2024-03-11T19:55:50.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/110: write d6/d19/d1e/f24 [490518,58186] 0 2024-03-11T19:55:50.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/111: mkdir d6/d28 0 2024-03-11T19:55:50.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/112: write d6/f1d [4204703,31614] 0 2024-03-11T19:55:50.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/113: getdents d6/d19 0 2024-03-11T19:55:50.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/114: chown d6/c8 8479788 1 2024-03-11T19:55:50.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/115: creat d6/d19/d1e/f29 x:0 0 0 2024-03-11T19:55:50.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/137: dread fa [0,4194304] 0 2024-03-11T19:55:50.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/146: dwrite d4/d7/f2c [0,4194304] 0 2024-03-11T19:55:50.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/147: chown d4/d7/d1f/d2b/d19 527778 1 2024-03-11T19:55:50.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/138: creat d10/d16/f2d x:0 0 0 2024-03-11T19:55:50.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/119: write fd [382165,125854] 0 2024-03-11T19:55:50.973 INFO:tasks.workunit.client.0.smithi107.stdout:0/120: getdents de/d11/d23 0 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/121: chown l8 549393 1 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/122: read - de/f17 zero size 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/116: symlink d6/d19/d1e/l2a 0 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/117: rename d6 to d6/d19/d2b 22 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/118: readlink d6/d19/d1e/l20 0 2024-03-11T19:55:50.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/119: stat d6/d19/d1e/d27 0 2024-03-11T19:55:50.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/120: rmdir d6/d19/d1e 39 2024-03-11T19:55:50.978 INFO:tasks.workunit.client.0.smithi107.stdout:1/121: mkdir d6/d19/d1e/d27/d2c 0 2024-03-11T19:55:50.978 INFO:tasks.workunit.client.0.smithi107.stdout:1/122: chown d6/d19/d1e/f24 189 1 2024-03-11T19:55:50.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/105: dwrite dd/d13/d18/f1c [0,4194304] 0 2024-03-11T19:55:51.005 INFO:tasks.workunit.client.0.smithi107.stdout:7/163: dwrite d6/da/d16/f2c [0,4194304] 0 2024-03-11T19:55:51.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:50 smithi122 ceph-mon[36397]: pgmap v75: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-03-11T19:55:51.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/99: dwrite d4/d1d/f1e [0,4194304] 0 2024-03-11T19:55:51.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/148: dwrite d4/d7/f2c [0,4194304] 0 2024-03-11T19:55:51.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/149: fdatasync d4/d7/f1d 0 2024-03-11T19:55:51.025 INFO:tasks.workunit.client.0.smithi107.stdout:4/150: mkdir d4/d7/d2f 0 2024-03-11T19:55:51.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/151: fsync d4/d7/f8 0 2024-03-11T19:55:51.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/152: creat d4/d7/d1f/d2b/d24/d26/d29/f30 x:0 0 0 2024-03-11T19:55:51.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/153: fsync d4/d7/f1d 0 2024-03-11T19:55:51.027 INFO:tasks.workunit.client.0.smithi107.stdout:0/123: write de/f16 [4151608,78829] 0 2024-03-11T19:55:51.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/154: creat d4/d7/d1f/d2b/d24/d26/d29/d2e/f31 x:0 0 0 2024-03-11T19:55:51.031 INFO:tasks.workunit.client.0.smithi107.stdout:4/155: creat d4/d7/d1f/d2b/d24/d26/d29/f32 x:0 0 0 2024-03-11T19:55:51.031 INFO:tasks.workunit.client.0.smithi107.stdout:0/124: unlink de/d11/c14 0 2024-03-11T19:55:51.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/139: dwrite fe [0,4194304] 0 2024-03-11T19:55:51.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/140: dread - d10/d16/d24/f27 zero size 2024-03-11T19:55:51.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:50 smithi145 ceph-mon[36730]: pgmap v75: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-03-11T19:55:51.036 INFO:tasks.workunit.client.0.smithi107.stdout:0/125: fdatasync fd 0 2024-03-11T19:55:51.036 INFO:tasks.workunit.client.0.smithi107.stdout:0/126: chown de/d11/l19 7456304 1 2024-03-11T19:55:51.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/141: mkdir d10/d16/d24/d2e 0 2024-03-11T19:55:51.038 INFO:tasks.workunit.client.0.smithi107.stdout:6/142: creat d10/d16/d24/d2e/f2f x:0 0 0 2024-03-11T19:55:51.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/143: link d10/l11 d10/d16/d2b/l30 0 2024-03-11T19:55:51.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/144: creat d10/d19/f31 x:0 0 0 2024-03-11T19:55:51.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:50 smithi107 ceph-mon[31712]: pgmap v75: 129 pgs: 129 active+clean; 122 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-03-11T19:55:51.042 INFO:tasks.workunit.client.0.smithi107.stdout:5/150: dwrite da/dd/f23 [0,4194304] 0 2024-03-11T19:55:51.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/151: chown da/dd/f23 12 1 2024-03-11T19:55:51.048 INFO:tasks.workunit.client.0.smithi107.stdout:2/130: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:51.059 INFO:tasks.workunit.client.0.smithi107.stdout:8/106: dwrite dd/de/d1b/f1d [0,4194304] 0 2024-03-11T19:55:51.059 INFO:tasks.workunit.client.0.smithi107.stdout:4/156: read d4/d7/f2c [1287639,87538] 0 2024-03-11T19:55:51.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/164: dwrite d6/d9/f8 [0,4194304] 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:1/123: dwrite d6/f10 [0,4194304] 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/145: dread d10/d16/f17 [0,4194304] 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:5/152: truncate da/dd/f23 1944615 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/146: creat d10/d16/d1f/f32 x:0 0 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/147: stat d10/d16/d24/d2e/f2f 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/148: creat d10/d16/f33 x:0 0 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/149: creat d10/d16/d24/d2e/f34 x:0 0 0 2024-03-11T19:55:51.062 INFO:tasks.workunit.client.0.smithi107.stdout:2/131: rename d1/ce to d1/d1a/d1e/d26/c29 0 2024-03-11T19:55:51.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/124: symlink d6/d19/d1e/d27/d2c/l2d 0 2024-03-11T19:55:51.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/127: write de/d11/f1a [1194874,91791] 0 2024-03-11T19:55:51.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/100: dwrite d4/f16 [0,4194304] 0 2024-03-11T19:55:51.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/101: chown d4/lf 414 1 2024-03-11T19:55:51.069 INFO:tasks.workunit.client.0.smithi107.stdout:9/102: write f1 [7448885,92428] 0 2024-03-11T19:55:51.069 INFO:tasks.workunit.client.0.smithi107.stdout:7/165: mknod d6/c2d 0 2024-03-11T19:55:51.072 INFO:tasks.workunit.client.0.smithi107.stdout:1/125: truncate d6/f1d 3783589 0 2024-03-11T19:55:51.073 INFO:tasks.workunit.client.0.smithi107.stdout:6/150: symlink d10/d19/l35 0 2024-03-11T19:55:51.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/132: unlink d1/f18 0 2024-03-11T19:55:51.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/133: write d1/f10 [1241536,90976] 0 2024-03-11T19:55:51.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/153: rename f9 to da/d14/d18/f2c 0 2024-03-11T19:55:51.090 INFO:tasks.workunit.client.0.smithi107.stdout:0/128: mknod de/d11/d23/d24/c27 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:5/154: creat da/dd/f2d x:0 0 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/134: mknod d1/d1a/d1e/d26/c2a 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/135: write d1/d1a/f23 [8719405,27499] 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:3/137: sync 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:3/138: write d1/d10/d21/f27 [24226,43429] 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:7/166: truncate d6/da/d16/f2c 1510374 0 2024-03-11T19:55:51.091 INFO:tasks.workunit.client.0.smithi107.stdout:7/167: write d6/da/d16/f2b [46502,43763] 0 2024-03-11T19:55:51.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/168: creat d6/da/d16/f2e x:0 0 0 2024-03-11T19:55:51.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/169: creat d6/f2f x:0 0 0 2024-03-11T19:55:51.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/151: dread d10/d16/f20 [0,4194304] 0 2024-03-11T19:55:51.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/129: rename de/c1f to de/c28 0 2024-03-11T19:55:51.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/136: dread d1/d1d/f24 [0,4194304] 0 2024-03-11T19:55:51.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/137: chown d1/d21/f28 0 1 2024-03-11T19:55:51.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/138: stat d1/fc 0 2024-03-11T19:55:51.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/107: dwrite f4 [8388608,4194304] 0 2024-03-11T19:55:51.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/130: creat de/d11/d23/d26/f29 x:0 0 0 2024-03-11T19:55:51.108 INFO:tasks.workunit.client.0.smithi107.stdout:0/131: stat f5 0 2024-03-11T19:55:51.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/108: creat dd/d16/f1e x:0 0 0 2024-03-11T19:55:51.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/109: chown dd/l12 284224532 1 2024-03-11T19:55:51.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/155: rmdir da 39 2024-03-11T19:55:51.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/156: write da/dd/f2d [70448,59620] 0 2024-03-11T19:55:51.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/170: getdents d6/da/d16 0 2024-03-11T19:55:51.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/171: dread - d6/f2f zero size 2024-03-11T19:55:51.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/172: creat d6/d9/f30 x:0 0 0 2024-03-11T19:55:51.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/173: chown d6/d9/d13/c12 68745640 1 2024-03-11T19:55:51.116 INFO:tasks.workunit.client.0.smithi107.stdout:9/103: dwrite f1 [4194304,4194304] 0 2024-03-11T19:55:51.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/132: getdents de 0 2024-03-11T19:55:51.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/139: dread d1/f15 [0,4194304] 0 2024-03-11T19:55:51.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/140: dread - d1/d1a/f20 zero size 2024-03-11T19:55:51.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/141: dread - d1/d1a/f20 zero size 2024-03-11T19:55:51.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/142: chown d1/d1a/d1e/d26 61 1 2024-03-11T19:55:51.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/143: chown d1/d1a/d1e 574976 1 2024-03-11T19:55:51.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/144: creat d1/f2b x:0 0 0 2024-03-11T19:55:51.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/145: stat d1/l27 0 2024-03-11T19:55:51.124 INFO:tasks.workunit.client.0.smithi107.stdout:0/133: symlink de/d11/d23/l2a 0 2024-03-11T19:55:51.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/146: creat d1/d1a/f2c x:0 0 0 2024-03-11T19:55:51.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/126: dwrite d6/fd [0,4194304] 0 2024-03-11T19:55:51.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/127: stat d6/fd 0 2024-03-11T19:55:51.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/128: chown l1 22 1 2024-03-11T19:55:51.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/157: dread da/dd/f2d [0,4194304] 0 2024-03-11T19:55:51.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/158: read da/dd/f11 [57340,114801] 0 2024-03-11T19:55:51.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/147: symlink d1/d1a/d1e/l2d 0 2024-03-11T19:55:51.132 INFO:tasks.workunit.client.0.smithi107.stdout:3/139: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:51.135 INFO:tasks.workunit.client.0.smithi107.stdout:5/159: getdents da/d14/d18/d1e 0 2024-03-11T19:55:51.135 INFO:tasks.workunit.client.0.smithi107.stdout:5/160: chown da/d14/d18/f27 60499 1 2024-03-11T19:55:51.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/161: write f5 [2362633,358] 0 2024-03-11T19:55:51.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/162: symlink da/d14/l2e 0 2024-03-11T19:55:51.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/163: link da/d14/l22 da/d14/d18/l2f 0 2024-03-11T19:55:51.148 INFO:tasks.workunit.client.0.smithi107.stdout:0/134: dread f3 [0,4194304] 0 2024-03-11T19:55:51.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/152: dwrite fe [4194304,4194304] 0 2024-03-11T19:55:51.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/153: creat d10/d16/d24/d2e/f36 x:0 0 0 2024-03-11T19:55:51.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/154: fdatasync d10/d19/f31 0 2024-03-11T19:55:51.155 INFO:tasks.workunit.client.0.smithi107.stdout:6/155: write fa [595418,123385] 0 2024-03-11T19:55:51.155 INFO:tasks.workunit.client.0.smithi107.stdout:6/156: dread - d10/d19/f31 zero size 2024-03-11T19:55:51.155 INFO:tasks.workunit.client.0.smithi107.stdout:6/157: creat d10/d16/f37 x:0 0 0 2024-03-11T19:55:51.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/104: dread d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:51.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/148: dread d1/fc [0,4194304] 0 2024-03-11T19:55:51.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/149: chown d1/d1a/d1e 1185979 1 2024-03-11T19:55:51.160 INFO:tasks.workunit.client.0.smithi107.stdout:6/158: rename d10/d16/d1f/d29 to d10/d19/d38 0 2024-03-11T19:55:51.161 INFO:tasks.workunit.client.0.smithi107.stdout:7/174: dwrite d6/da/d16/f2e [0,4194304] 0 2024-03-11T19:55:51.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/175: readlink d6/da/l27 0 2024-03-11T19:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/159: mkdir d10/d16/d2b/d39 0 2024-03-11T19:55:51.168 INFO:tasks.workunit.client.0.smithi107.stdout:9/105: rename d4/d10 to d4/d22 0 2024-03-11T19:55:51.168 INFO:tasks.workunit.client.0.smithi107.stdout:9/106: chown d4/d1d/d1f 4 1 2024-03-11T19:55:51.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/110: dwrite dd/de/d1b/f1d [4194304,4194304] 0 2024-03-11T19:55:51.171 INFO:tasks.workunit.client.0.smithi107.stdout:9/107: unlink d4/d1d/d1f/l20 0 2024-03-11T19:55:51.172 INFO:tasks.workunit.client.0.smithi107.stdout:1/129: dwrite d6/d19/d1e/f29 [0,4194304] 0 2024-03-11T19:55:51.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/111: symlink dd/de/d1b/l1f 0 2024-03-11T19:55:51.175 INFO:tasks.workunit.client.0.smithi107.stdout:9/108: link d4/c5 d4/d1c/c23 0 2024-03-11T19:55:51.177 INFO:tasks.workunit.client.0.smithi107.stdout:3/140: dread d1/d10/d21/f27 [0,4194304] 0 2024-03-11T19:55:51.177 INFO:tasks.workunit.client.0.smithi107.stdout:3/141: dread - d1/d10/d13/f2c zero size 2024-03-11T19:55:51.177 INFO:tasks.workunit.client.0.smithi107.stdout:3/142: write d1/d10/d13/f2c [437356,108873] 0 2024-03-11T19:55:51.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/112: mkdir dd/de/d1b/d20 0 2024-03-11T19:55:51.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/113: fdatasync dd/d16/f1e 0 2024-03-11T19:55:51.179 INFO:tasks.workunit.client.0.smithi107.stdout:1/130: link d6/f14 d6/d28/f2e 0 2024-03-11T19:55:51.180 INFO:tasks.workunit.client.0.smithi107.stdout:3/143: mknod d1/d12/d1a/c2e 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:1/131: symlink d6/d28/l2f 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:1/132: stat d6/la 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/114: creat dd/f21 x:0 0 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/144: getdents d1/d19 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/145: creat d1/d12/f2f x:0 0 0 2024-03-11T19:55:51.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/146: truncate d1/d12/f14 5032021 0 2024-03-11T19:55:51.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/147: chown d1/d12/d1a/c2e 160 1 2024-03-11T19:55:51.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/148: chown d1/d19/l24 1170674613 1 2024-03-11T19:55:51.182 INFO:tasks.workunit.client.0.smithi107.stdout:1/133: unlink d6/c17 0 2024-03-11T19:55:51.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/149: rename d1/d12/f14 to d1/d10/d26/f30 0 2024-03-11T19:55:51.184 INFO:tasks.workunit.client.0.smithi107.stdout:2/150: write d1/d1a/f23 [7224756,128193] 0 2024-03-11T19:55:51.184 INFO:tasks.workunit.client.0.smithi107.stdout:1/134: creat d6/d19/d1e/d27/d2c/f30 x:0 0 0 2024-03-11T19:55:51.186 INFO:tasks.workunit.client.0.smithi107.stdout:2/151: unlink d1/l16 0 2024-03-11T19:55:51.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/115: fsync dd/f21 0 2024-03-11T19:55:51.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/116: fsync dd/d13/d18/f1a 0 2024-03-11T19:55:51.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/117: chown c8 17 1 2024-03-11T19:55:51.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/160: write d10/d19/f23 [4034529,16125] 0 2024-03-11T19:55:51.190 INFO:tasks.workunit.client.0.smithi107.stdout:9/109: dread d4/f11 [0,4194304] 0 2024-03-11T19:55:51.190 INFO:tasks.workunit.client.0.smithi107.stdout:8/118: mkdir dd/d13/d22 0 2024-03-11T19:55:51.192 INFO:tasks.workunit.client.0.smithi107.stdout:3/150: truncate d1/d10/d13/f2c 23757 0 2024-03-11T19:55:51.192 INFO:tasks.workunit.client.0.smithi107.stdout:3/151: chown d1/d12/c18 0 1 2024-03-11T19:55:51.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/152: getdents d1/d12/d1a 0 2024-03-11T19:55:51.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/153: rename d1 to d1/d12/d31 22 2024-03-11T19:55:51.196 INFO:tasks.workunit.client.0.smithi107.stdout:3/154: creat d1/d19/f32 x:0 0 0 2024-03-11T19:55:51.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/152: dread d1/d1d/f24 [0,4194304] 0 2024-03-11T19:55:51.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/176: dwrite d6/f22 [0,4194304] 0 2024-03-11T19:55:51.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/177: chown d6/d9/f23 98578358 1 2024-03-11T19:55:51.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/178: creat d6/da/f31 x:0 0 0 2024-03-11T19:55:51.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/153: creat d1/d21/f2e x:0 0 0 2024-03-11T19:55:51.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/154: fdatasync d1/f15 0 2024-03-11T19:55:51.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/155: truncate d1/f2b 58004 0 2024-03-11T19:55:51.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/179: link d6/d9/c1d d6/c32 0 2024-03-11T19:55:51.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/180: readlink d6/da/l27 0 2024-03-11T19:55:51.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/181: readlink d6/da/l27 0 2024-03-11T19:55:51.212 INFO:tasks.workunit.client.0.smithi107.stdout:1/135: link l1 d6/d19/d1e/l31 0 2024-03-11T19:55:51.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/136: mkdir d6/d19/d1e/d27/d2c/d32 0 2024-03-11T19:55:51.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/137: truncate d6/d28/f2e 492004 0 2024-03-11T19:55:51.216 INFO:tasks.workunit.client.0.smithi107.stdout:1/138: rename d6/l13 to d6/d28/l33 0 2024-03-11T19:55:51.216 INFO:tasks.workunit.client.0.smithi107.stdout:1/139: creat d6/d19/d1e/d27/d2c/f34 x:0 0 0 2024-03-11T19:55:51.220 INFO:tasks.workunit.client.0.smithi107.stdout:1/140: link d6/la d6/l35 0 2024-03-11T19:55:51.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/161: dwrite fc [4194304,4194304] 0 2024-03-11T19:55:51.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/162: write d10/d16/d24/d2e/f34 [849681,117095] 0 2024-03-11T19:55:51.231 INFO:tasks.workunit.client.0.smithi107.stdout:8/119: write dd/de/d1b/f1d [6106668,40427] 0 2024-03-11T19:55:51.233 INFO:tasks.workunit.client.0.smithi107.stdout:6/163: getdents d10/d19 0 2024-03-11T19:55:51.233 INFO:tasks.workunit.client.0.smithi107.stdout:6/164: creat d10/d16/f3a x:0 0 0 2024-03-11T19:55:51.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/165: mkdir d10/d19/d3b 0 2024-03-11T19:55:51.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/166: unlink d10/l11 0 2024-03-11T19:55:51.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/167: fdatasync fe 0 2024-03-11T19:55:51.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/141: write d6/f10 [3604605,99181] 0 2024-03-11T19:55:51.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/142: chown d6/d28/f2e 23195234 1 2024-03-11T19:55:51.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/155: dwrite d1/d10/d26/d17/f1f [0,4194304] 0 2024-03-11T19:55:51.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/156: chown d1/d12/c18 0 1 2024-03-11T19:55:51.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/168: dwrite fc [4194304,4194304] 0 2024-03-11T19:55:51.278 INFO:tasks.workunit.client.0.smithi107.stdout:8/120: write dd/d13/d18/f1c [3847937,26407] 0 2024-03-11T19:55:51.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/157: read d1/d10/d26/f30 [4946488,104141] 0 2024-03-11T19:55:51.280 INFO:tasks.workunit.client.0.smithi107.stdout:1/143: rename l1 to d6/d19/l36 0 2024-03-11T19:55:51.280 INFO:tasks.workunit.client.0.smithi107.stdout:1/144: stat d6/l1a 0 2024-03-11T19:55:51.280 INFO:tasks.workunit.client.0.smithi107.stdout:1/145: write d6/f15 [4194595,16792] 0 2024-03-11T19:55:51.280 INFO:tasks.workunit.client.0.smithi107.stdout:1/146: dread - d6/d19/d1e/f26 zero size 2024-03-11T19:55:51.281 INFO:tasks.workunit.client.0.smithi107.stdout:1/147: mknod d6/d19/d1e/d27/d2c/d32/c37 0 2024-03-11T19:55:51.281 INFO:tasks.workunit.client.0.smithi107.stdout:1/148: dread - d6/f16 zero size 2024-03-11T19:55:51.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/169: rename fe to d10/d16/d24/d2e/f3c 0 2024-03-11T19:55:51.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/158: read d1/fa [174296,63476] 0 2024-03-11T19:55:51.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/159: symlink d1/d19/d2d/l33 0 2024-03-11T19:55:51.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/160: stat d1/d19 0 2024-03-11T19:55:51.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/110: dwrite d4/d9/f14 [0,4194304] 0 2024-03-11T19:55:51.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/149: getdents d6/d19/d1e/d27/d2c/d32 0 2024-03-11T19:55:51.286 INFO:tasks.workunit.client.0.smithi107.stdout:1/150: creat d6/d19/d1e/d27/f38 x:0 0 0 2024-03-11T19:55:51.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/182: dread d6/d9/d13/d1b/f1c [0,4194304] 0 2024-03-11T19:55:51.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/111: symlink d4/d22/d17/l24 0 2024-03-11T19:55:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/183: link d6/da/d16/c18 d6/da/c33 0 2024-03-11T19:55:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/121: dread f4 [8388608,4194304] 0 2024-03-11T19:55:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:4/157: sync 2024-03-11T19:55:51.303 INFO:tasks.workunit.client.0.smithi107.stdout:4/158: chown d4/d7/db 238 1 2024-03-11T19:55:51.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/159: readlink d4/d7/d1f/d2b/d19/l1c 0 2024-03-11T19:55:51.304 INFO:tasks.workunit.client.0.smithi107.stdout:3/161: link d1/lc d1/d10/d13/l34 0 2024-03-11T19:55:51.304 INFO:tasks.workunit.client.0.smithi107.stdout:3/162: fsync d1/d10/d26/f30 0 2024-03-11T19:55:51.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/163: mkdir d1/d35 0 2024-03-11T19:55:51.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/156: dwrite d1/d1a/f23 [0,4194304] 0 2024-03-11T19:55:51.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/157: creat d1/d1a/f2f x:0 0 0 2024-03-11T19:55:51.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/158: write d1/d21/f2e [593193,17882] 0 2024-03-11T19:55:51.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/159: creat d1/d1a/d1e/d26/f30 x:0 0 0 2024-03-11T19:55:51.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/160: truncate d1/d1a/f20 7072 0 2024-03-11T19:55:51.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/170: dwrite d10/d19/f31 [0,4194304] 0 2024-03-11T19:55:51.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/122: dwrite dd/d13/d18/f1a [0,4194304] 0 2024-03-11T19:55:51.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/123: write f3 [1175478,116562] 0 2024-03-11T19:55:51.345 INFO:tasks.workunit.client.0.smithi107.stdout:4/160: dread d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:51.346 INFO:tasks.workunit.client.0.smithi107.stdout:4/161: dread - d4/d7/d1f/d2b/d24/d26/d29/f2d zero size 2024-03-11T19:55:51.348 INFO:tasks.workunit.client.0.smithi107.stdout:6/171: rmdir d10 39 2024-03-11T19:55:51.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/124: mknod dd/d13/d18/c23 0 2024-03-11T19:55:51.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/162: rmdir d4/d7/d1f/d2b/d19/d27 0 2024-03-11T19:55:51.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/163: stat d4/d7/d1f/l25 0 2024-03-11T19:55:51.351 INFO:tasks.workunit.client.0.smithi107.stdout:6/172: unlink d10/d16/d1f/f32 0 2024-03-11T19:55:51.352 INFO:tasks.workunit.client.0.smithi107.stdout:4/164: mkdir d4/d7/d2f/d33 0 2024-03-11T19:55:51.352 INFO:tasks.workunit.client.0.smithi107.stdout:4/165: stat d4/d7/db/lf 0 2024-03-11T19:55:51.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/112: write d4/d22/d17/f19 [2487924,38622] 0 2024-03-11T19:55:51.355 INFO:tasks.workunit.client.0.smithi107.stdout:1/151: dwrite d6/f1d [0,4194304] 0 2024-03-11T19:55:51.355 INFO:tasks.workunit.client.0.smithi107.stdout:1/152: chown d6/l35 96 1 2024-03-11T19:55:51.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/153: mkdir d6/d19/d39 0 2024-03-11T19:55:51.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/154: readlink d6/d19/d1e/l1f 0 2024-03-11T19:55:51.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/113: mknod d4/d22/c25 0 2024-03-11T19:55:51.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/114: mknod d4/d1d/d1f/c26 0 2024-03-11T19:55:51.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/164: dwrite d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:55:51.373 INFO:tasks.workunit.client.0.smithi107.stdout:2/161: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:51.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/165: rmdir d1/d35 0 2024-03-11T19:55:51.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/166: readlink d1/l8 0 2024-03-11T19:55:51.375 INFO:tasks.workunit.client.0.smithi107.stdout:1/155: dread d6/f14 [0,4194304] 0 2024-03-11T19:55:51.381 INFO:tasks.workunit.client.0.smithi107.stdout:2/162: rmdir d1/d1a 39 2024-03-11T19:55:51.381 INFO:tasks.workunit.client.0.smithi107.stdout:2/163: fsync d1/f2b 0 2024-03-11T19:55:51.381 INFO:tasks.workunit.client.0.smithi107.stdout:2/164: write d1/f6 [2216790,60655] 0 2024-03-11T19:55:51.381 INFO:tasks.workunit.client.0.smithi107.stdout:1/156: rename d6/d28/l33 to d6/d19/d1e/d27/d2c/d32/l3a 0 2024-03-11T19:55:51.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/165: rmdir d1/d1a/d1e/d26 39 2024-03-11T19:55:51.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/157: read - d6/d19/d1e/d27/d2c/f34 zero size 2024-03-11T19:55:51.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/166: mknod d1/d1d/c31 0 2024-03-11T19:55:51.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/167: readlink d1/l17 0 2024-03-11T19:55:51.383 INFO:tasks.workunit.client.0.smithi107.stdout:1/158: symlink d6/d19/d1e/d27/l3b 0 2024-03-11T19:55:51.383 INFO:tasks.workunit.client.0.smithi107.stdout:1/159: write d6/f10 [1223647,69469] 0 2024-03-11T19:55:51.384 INFO:tasks.workunit.client.0.smithi107.stdout:1/160: creat d6/d19/d1e/f3c x:0 0 0 2024-03-11T19:55:51.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/161: write d6/d19/d1e/f26 [519295,58273] 0 2024-03-11T19:55:51.387 INFO:tasks.workunit.client.0.smithi107.stdout:2/168: rename d1/d1a/f23 to d1/d1a/d1e/d26/f32 0 2024-03-11T19:55:51.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/125: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:51.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/126: fdatasync dd/d13/d18/f1c 0 2024-03-11T19:55:51.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/127: chown dd/d13/d22 1271 1 2024-03-11T19:55:51.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/173: dwrite d10/d16/f37 [0,4194304] 0 2024-03-11T19:55:51.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/174: dread - d10/d16/f2d zero size 2024-03-11T19:55:51.408 INFO:tasks.workunit.client.0.smithi107.stdout:6/175: mkdir d10/d3d 0 2024-03-11T19:55:51.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/166: dwrite d4/d7/f22 [0,4194304] 0 2024-03-11T19:55:51.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/167: symlink d4/d7/db/l34 0 2024-03-11T19:55:51.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/168: mknod d4/d7/c35 0 2024-03-11T19:55:51.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/128: dread f4 [8388608,4194304] 0 2024-03-11T19:55:51.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/129: chown dd/de/d1b/d20 16530529 1 2024-03-11T19:55:51.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/130: write dd/d13/d18/f1c [659252,60590] 0 2024-03-11T19:55:51.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/169: mkdir d4/d7/d1f/d2b/d24/d26/d29/d36 0 2024-03-11T19:55:51.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/167: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:51.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/168: write d1/d10/d21/f28 [844508,112855] 0 2024-03-11T19:55:51.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/169: readlink d1/ld 0 2024-03-11T19:55:51.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/170: readlink d1/d10/l15 0 2024-03-11T19:55:51.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/171: chown d1/lc 60316 1 2024-03-11T19:55:51.427 INFO:tasks.workunit.client.0.smithi107.stdout:8/131: unlink dd/de/d1b/l1f 0 2024-03-11T19:55:51.427 INFO:tasks.workunit.client.0.smithi107.stdout:8/132: fdatasync dd/d13/d18/f1a 0 2024-03-11T19:55:51.427 INFO:tasks.workunit.client.0.smithi107.stdout:8/133: write dd/d16/f1e [347732,28287] 0 2024-03-11T19:55:51.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/134: getdents dd/de 0 2024-03-11T19:55:51.435 INFO:tasks.workunit.client.0.smithi107.stdout:2/169: write d1/f10 [4136608,19512] 0 2024-03-11T19:55:51.435 INFO:tasks.workunit.client.0.smithi107.stdout:1/162: dwrite d6/fd [0,4194304] 0 2024-03-11T19:55:51.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/170: mkdir d1/d1a/d1e/d26/d33 0 2024-03-11T19:55:51.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/171: mkdir d1/d1a/d34 0 2024-03-11T19:55:51.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/172: fdatasync d1/d1a/f20 0 2024-03-11T19:55:51.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/173: chown d1/l27 1 1 2024-03-11T19:55:51.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/172: dread d1/d10/d26/d17/f1f [0,4194304] 0 2024-03-11T19:55:51.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/174: truncate d1/f15 621989 0 2024-03-11T19:55:51.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/175: chown d1/d1a/d1e/d26/d33 228356371 1 2024-03-11T19:55:51.452 INFO:tasks.workunit.client.0.smithi107.stdout:2/176: mkdir d1/d21/d35 0 2024-03-11T19:55:51.453 INFO:tasks.workunit.client.0.smithi107.stdout:2/177: creat d1/f36 x:0 0 0 2024-03-11T19:55:51.453 INFO:tasks.workunit.client.0.smithi107.stdout:2/178: write d1/f2b [824367,24708] 0 2024-03-11T19:55:51.456 INFO:tasks.workunit.client.0.smithi107.stdout:2/179: rename d1/l5 to d1/d1a/d1e/l37 0 2024-03-11T19:55:51.457 INFO:tasks.workunit.client.0.smithi107.stdout:2/180: write d1/f10 [3073124,52766] 0 2024-03-11T19:55:51.472 INFO:tasks.workunit.client.0.smithi107.stdout:8/135: write f4 [8195355,63821] 0 2024-03-11T19:55:51.472 INFO:tasks.workunit.client.0.smithi107.stdout:8/136: chown f3 68 1 2024-03-11T19:55:51.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/181: read d1/d21/f2e [484659,16516] 0 2024-03-11T19:55:51.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/176: dwrite fa [0,4194304] 0 2024-03-11T19:55:51.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/170: dwrite d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:51.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/182: mkdir d1/d38 0 2024-03-11T19:55:51.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/137: read f3 [996345,22222] 0 2024-03-11T19:55:51.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/138: chown dd/de/d1b 2343 1 2024-03-11T19:55:51.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/139: read f3 [885225,50873] 0 2024-03-11T19:55:51.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/177: dread d10/d16/d1f/f2a [0,4194304] 0 2024-03-11T19:55:51.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/171: getdents d4 0 2024-03-11T19:55:51.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/172: dread - d4/d7/d1f/d2b/d19/f2a zero size 2024-03-11T19:55:51.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/173: readlink d4/d7/d1f/l25 0 2024-03-11T19:55:51.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/174: write d4/d7/f17 [1778,12007] 0 2024-03-11T19:55:51.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/183: mknod d1/d38/c39 0 2024-03-11T19:55:51.482 INFO:tasks.workunit.client.0.smithi107.stdout:5/164: sync 2024-03-11T19:55:51.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/184: sync 2024-03-11T19:55:51.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/135: sync 2024-03-11T19:55:51.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/136: fsync de/f17 0 2024-03-11T19:55:51.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/115: sync 2024-03-11T19:55:51.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/116: creat d4/f27 x:0 0 0 2024-03-11T19:55:51.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/140: mkdir dd/de/d24 0 2024-03-11T19:55:51.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/175: symlink d4/d7/d1f/d2b/d24/d26/d29/d36/l37 0 2024-03-11T19:55:51.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/185: symlink d6/d9/l34 0 2024-03-11T19:55:51.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/186: truncate d6/f29 717544 0 2024-03-11T19:55:51.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/187: chown d6/d9/d13/c12 1 1 2024-03-11T19:55:51.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/178: creat d10/d16/d2b/f3e x:0 0 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/184: symlink d1/d38/l3a 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/163: dwrite d6/d19/d1e/f3c [0,4194304] 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/176: creat d4/d7/d1f/d2b/d24/d26/f38 x:0 0 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/177: chown d4/d7/d1f/d2b/l16 52 1 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/178: readlink d4/d7/db/l10 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/188: unlink d6/da/f31 0 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/189: chown d6/da/f1f 1 1 2024-03-11T19:55:51.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/185: write d1/d1a/f2f [404536,89281] 0 2024-03-11T19:55:51.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/137: link de/c15 de/d11/c2b 0 2024-03-11T19:55:51.493 INFO:tasks.workunit.client.0.smithi107.stdout:6/179: symlink d10/d16/d24/l3f 0 2024-03-11T19:55:51.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/138: creat de/d11/f2c x:0 0 0 2024-03-11T19:55:51.497 INFO:tasks.workunit.client.0.smithi107.stdout:6/180: mkdir d10/d19/d38/d40 0 2024-03-11T19:55:51.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/190: rmdir d6/d9/d13 39 2024-03-11T19:55:51.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/191: truncate d6/da/d16/f1a 1801800 0 2024-03-11T19:55:51.498 INFO:tasks.workunit.client.0.smithi107.stdout:0/139: creat de/d11/d23/d24/f2d x:0 0 0 2024-03-11T19:55:51.498 INFO:tasks.workunit.client.0.smithi107.stdout:0/140: chown de/d11/f2c 855165647 1 2024-03-11T19:55:51.501 INFO:tasks.workunit.client.0.smithi107.stdout:0/141: truncate de/d11/f1a 647411 0 2024-03-11T19:55:51.501 INFO:tasks.workunit.client.0.smithi107.stdout:0/142: stat de/d11/f2c 0 2024-03-11T19:55:51.503 INFO:tasks.workunit.client.0.smithi107.stdout:0/143: creat de/d11/d23/f2e x:0 0 0 2024-03-11T19:55:51.505 INFO:tasks.workunit.client.0.smithi107.stdout:0/144: creat de/f2f x:0 0 0 2024-03-11T19:55:51.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/117: write d4/d9/f14 [1838986,84180] 0 2024-03-11T19:55:51.532 INFO:tasks.workunit.client.0.smithi107.stdout:1/164: write d6/f1d [4059726,8532] 0 2024-03-11T19:55:51.532 INFO:tasks.workunit.client.0.smithi107.stdout:1/165: readlink d6/d28/l2f 0 2024-03-11T19:55:51.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/118: mkdir d4/d1d/d1f/d28 0 2024-03-11T19:55:51.540 INFO:tasks.workunit.client.0.smithi107.stdout:9/119: symlink d4/d1d/d1f/d28/l29 0 2024-03-11T19:55:51.540 INFO:tasks.workunit.client.0.smithi107.stdout:9/120: creat d4/f2a x:0 0 0 2024-03-11T19:55:51.542 INFO:tasks.workunit.client.0.smithi107.stdout:9/121: symlink d4/l2b 0 2024-03-11T19:55:51.543 INFO:tasks.workunit.client.0.smithi107.stdout:9/122: mkdir d4/d1d/d1f/d28/d2c 0 2024-03-11T19:55:51.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/141: dread dd/d16/f19 [0,4194304] 0 2024-03-11T19:55:51.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/142: readlink dd/l12 0 2024-03-11T19:55:51.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/179: dwrite d4/d7/db/f21 [0,4194304] 0 2024-03-11T19:55:51.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/180: stat d4/d7/d2f/d33 0 2024-03-11T19:55:51.549 INFO:tasks.workunit.client.0.smithi107.stdout:5/165: dwrite da/d14/d18/f2c [0,4194304] 0 2024-03-11T19:55:51.552 INFO:tasks.workunit.client.0.smithi107.stdout:4/181: symlink d4/d7/d1f/l39 0 2024-03-11T19:55:51.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/166: link da/d14/l2e da/dd/l30 0 2024-03-11T19:55:51.560 INFO:tasks.workunit.client.0.smithi107.stdout:4/182: truncate d4/d7/db/f21 2658601 0 2024-03-11T19:55:51.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/183: mkdir d4/d7/d2f/d33/d3a 0 2024-03-11T19:55:51.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/167: write da/dd/f23 [780902,77057] 0 2024-03-11T19:55:51.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/168: stat da/dd/l20 0 2024-03-11T19:55:51.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/169: mkdir da/d31 0 2024-03-11T19:55:51.568 INFO:tasks.workunit.client.0.smithi107.stdout:5/170: creat da/f32 x:0 0 0 2024-03-11T19:55:51.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/181: dread d10/d19/f31 [0,4194304] 0 2024-03-11T19:55:51.575 INFO:tasks.workunit.client.0.smithi107.stdout:6/182: fsync d10/d16/f3a 0 2024-03-11T19:55:51.575 INFO:tasks.workunit.client.0.smithi107.stdout:0/145: dwrite f5 [0,4194304] 0 2024-03-11T19:55:51.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/171: dread da/d14/d18/f1c [0,4194304] 0 2024-03-11T19:55:51.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/192: read d6/da/d16/f2e [2510118,57051] 0 2024-03-11T19:55:51.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/193: getdents d6/da/d16 0 2024-03-11T19:55:51.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/194: stat d6/f29 0 2024-03-11T19:55:51.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/195: stat d6/da/f1f 0 2024-03-11T19:55:51.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/196: symlink d6/l35 0 2024-03-11T19:55:51.583 INFO:tasks.workunit.client.0.smithi107.stdout:5/172: rename da/d14/c24 to da/c33 0 2024-03-11T19:55:51.583 INFO:tasks.workunit.client.0.smithi107.stdout:5/173: fdatasync da/fb 0 2024-03-11T19:55:51.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/183: symlink d10/d16/d2b/l41 0 2024-03-11T19:55:51.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/184: write d10/d16/d1f/f28 [371357,69030] 0 2024-03-11T19:55:51.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/197: rename d6/d9/f23 to d6/da/d16/f36 0 2024-03-11T19:55:51.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/198: rmdir d6/d9/d13 39 2024-03-11T19:55:51.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/199: symlink d6/d9/d13/d1b/l37 0 2024-03-11T19:55:51.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/200: truncate d6/f2f 996650 0 2024-03-11T19:55:51.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/201: mknod d6/d9/c38 0 2024-03-11T19:55:51.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/202: chown d6/da/d16/c18 1621468 1 2024-03-11T19:55:51.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/203: read - d6/d21/f2a zero size 2024-03-11T19:55:51.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/204: creat d6/d9/f39 x:0 0 0 2024-03-11T19:55:51.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/184: dread d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:51.593 INFO:tasks.workunit.client.0.smithi107.stdout:4/185: truncate d4/fc 2159765 0 2024-03-11T19:55:51.593 INFO:tasks.workunit.client.0.smithi107.stdout:4/186: getdents d4/d7/d2f/d33/d3a 0 2024-03-11T19:55:51.599 INFO:tasks.workunit.client.0.smithi107.stdout:8/143: read dd/d16/f19 [1112810,4568] 0 2024-03-11T19:55:51.600 INFO:tasks.workunit.client.0.smithi107.stdout:4/187: rename d4/d7/d1f/d2b/d24/d26/d29/d2e to d4/d7/d2f/d3b 0 2024-03-11T19:55:51.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/144: mknod dd/d16/c25 0 2024-03-11T19:55:51.603 INFO:tasks.workunit.client.0.smithi107.stdout:8/145: creat dd/de/d1b/d20/f26 x:0 0 0 2024-03-11T19:55:51.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/185: dread d10/d19/f23 [0,4194304] 0 2024-03-11T19:55:51.605 INFO:tasks.workunit.client.0.smithi107.stdout:8/146: truncate f4 7856487 0 2024-03-11T19:55:51.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/123: dwrite d4/d22/d17/f1b [0,4194304] 0 2024-03-11T19:55:51.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/124: chown d4 786951788 1 2024-03-11T19:55:51.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/125: chown d4/d1d/f1e 78926 1 2024-03-11T19:55:51.620 INFO:tasks.workunit.client.0.smithi107.stdout:8/147: read dd/de/f15 [309510,9460] 0 2024-03-11T19:55:51.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/148: mknod dd/de/d1b/c27 0 2024-03-11T19:55:51.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/174: dwrite da/d14/d19/f1d [0,4194304] 0 2024-03-11T19:55:51.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/188: dread d4/d7/db/f21 [0,4194304] 0 2024-03-11T19:55:51.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/189: dread - d4/d7/d1f/d2b/d24/d26/d29/f32 zero size 2024-03-11T19:55:51.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/190: mkdir d4/d7/d2f/d3c 0 2024-03-11T19:55:51.632 INFO:tasks.workunit.client.0.smithi107.stdout:9/126: dread d4/f11 [0,4194304] 0 2024-03-11T19:55:51.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/191: symlink d4/d7/d2f/d33/l3d 0 2024-03-11T19:55:51.637 INFO:tasks.workunit.client.0.smithi107.stdout:9/127: creat d4/d22/f2d x:0 0 0 2024-03-11T19:55:51.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/128: unlink d4/d1d/f1e 0 2024-03-11T19:55:51.645 INFO:tasks.workunit.client.0.smithi107.stdout:0/146: dwrite f3 [0,4194304] 0 2024-03-11T19:55:51.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/205: dwrite d6/f2f [0,4194304] 0 2024-03-11T19:55:51.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/206: creat d6/d21/f3a x:0 0 0 2024-03-11T19:55:51.652 INFO:tasks.workunit.client.0.smithi107.stdout:7/207: truncate d6/d9/d13/d1b/f1c 2516750 0 2024-03-11T19:55:51.652 INFO:tasks.workunit.client.0.smithi107.stdout:7/208: rename d6/d9 to d6/d9/d13/d3b 22 2024-03-11T19:55:51.652 INFO:tasks.workunit.client.0.smithi107.stdout:7/209: fdatasync d6/d9/f30 0 2024-03-11T19:55:51.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/210: stat d6/d9/f8 0 2024-03-11T19:55:51.654 INFO:tasks.workunit.client.0.smithi107.stdout:7/211: creat d6/d9/d13/d1b/f3c x:0 0 0 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:7/212: getdents d6/da/d16 0 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/175: dread f4 [0,4194304] 0 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/176: fsync da/d14/d18/f1c 0 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/177: stat da/d14 0 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/178: chown da/dd/f23 1 1 2024-03-11T19:55:51.656 INFO:tasks.workunit.client.0.smithi107.stdout:7/213: write d6/da/f1e [2581813,49254] 0 2024-03-11T19:55:51.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/179: link da/d14/c17 da/d14/d18/d1e/c34 0 2024-03-11T19:55:51.659 INFO:tasks.workunit.client.0.smithi107.stdout:6/186: dwrite d10/d16/f33 [0,4194304] 0 2024-03-11T19:55:51.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/187: write d10/d16/d2b/f3e [794003,96899] 0 2024-03-11T19:55:51.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/180: rmdir da/d14/d19 39 2024-03-11T19:55:51.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/181: write da/dd/f23 [1781826,51666] 0 2024-03-11T19:55:51.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/149: dread f4 [0,4194304] 0 2024-03-11T19:55:51.666 INFO:tasks.workunit.client.0.smithi107.stdout:5/182: link f4 da/d14/f35 0 2024-03-11T19:55:51.673 INFO:tasks.workunit.client.0.smithi107.stdout:8/150: mkdir dd/de/d24/d28 0 2024-03-11T19:55:51.673 INFO:tasks.workunit.client.0.smithi107.stdout:8/151: fdatasync dd/d16/f1e 0 2024-03-11T19:55:51.673 INFO:tasks.workunit.client.0.smithi107.stdout:8/152: chown f4 207708192 1 2024-03-11T19:55:51.674 INFO:tasks.workunit.client.0.smithi107.stdout:5/183: link da/f1a da/f36 0 2024-03-11T19:55:51.676 INFO:tasks.workunit.client.0.smithi107.stdout:5/184: mknod da/d14/d18/d1e/c37 0 2024-03-11T19:55:51.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/173: sync 2024-03-11T19:55:51.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/174: truncate d1/d19/f32 492086 0 2024-03-11T19:55:51.686 INFO:tasks.workunit.client.0.smithi107.stdout:5/185: dread da/d14/d18/f1c [0,4194304] 0 2024-03-11T19:55:51.688 INFO:tasks.workunit.client.0.smithi107.stdout:5/186: unlink da/d14/c25 0 2024-03-11T19:55:51.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/187: mkdir da/d14/d38 0 2024-03-11T19:55:51.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/188: mknod da/d31/c39 0 2024-03-11T19:55:51.691 INFO:tasks.workunit.client.0.smithi107.stdout:5/189: getdents da/d14/d18/d1e 0 2024-03-11T19:55:51.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/175: dread d1/f5 [4194304,4194304] 0 2024-03-11T19:55:51.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/176: fsync d1/d12/f2f 0 2024-03-11T19:55:51.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/190: truncate f4 3161917 0 2024-03-11T19:55:51.697 INFO:tasks.workunit.client.0.smithi107.stdout:7/214: dwrite d6/f29 [0,4194304] 0 2024-03-11T19:55:51.702 INFO:tasks.workunit.client.0.smithi107.stdout:7/215: fdatasync d6/d9/d13/d1b/f1c 0 2024-03-11T19:55:51.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/147: dwrite de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:51.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/129: dread d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:51.715 INFO:tasks.workunit.client.0.smithi107.stdout:6/188: dwrite d10/d16/f37 [0,4194304] 0 2024-03-11T19:55:51.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/130: mknod d4/d1d/c2e 0 2024-03-11T19:55:51.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/191: dread da/dd/f23 [0,4194304] 0 2024-03-11T19:55:51.716 INFO:tasks.workunit.client.0.smithi107.stdout:9/131: creat d4/d1d/d1f/f2f x:0 0 0 2024-03-11T19:55:51.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/153: dwrite f3 [0,4194304] 0 2024-03-11T19:55:51.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/192: dread da/dd/f28 [4194304,4194304] 0 2024-03-11T19:55:51.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/154: unlink dd/d16/c25 0 2024-03-11T19:55:51.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/193: creat da/d14/f3a x:0 0 0 2024-03-11T19:55:51.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/177: dwrite d1/f5 [4194304,4194304] 0 2024-03-11T19:55:51.735 INFO:tasks.workunit.client.0.smithi107.stdout:9/132: dread d4/f11 [0,4194304] 0 2024-03-11T19:55:51.735 INFO:tasks.workunit.client.0.smithi107.stdout:5/194: write da/dd/f13 [976324,68395] 0 2024-03-11T19:55:51.735 INFO:tasks.workunit.client.0.smithi107.stdout:9/133: write d4/d22/d17/f1a [1031265,26130] 0 2024-03-11T19:55:51.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/178: symlink d1/d12/d1a/l36 0 2024-03-11T19:55:51.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/195: rmdir da/d14/d19 39 2024-03-11T19:55:51.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/189: read d10/d16/f1c [1174264,82181] 0 2024-03-11T19:55:51.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/190: write fc [9204446,21718] 0 2024-03-11T19:55:51.744 INFO:tasks.workunit.client.0.smithi107.stdout:6/191: creat d10/d16/d1f/f42 x:0 0 0 2024-03-11T19:55:51.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/148: write f3 [1182140,77360] 0 2024-03-11T19:55:51.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/149: rmdir de/d11/d23/d24 39 2024-03-11T19:55:51.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/150: stat de/d11/f2c 0 2024-03-11T19:55:51.759 INFO:tasks.workunit.client.0.smithi107.stdout:0/151: mknod de/c30 0 2024-03-11T19:55:51.760 INFO:tasks.workunit.client.0.smithi107.stdout:0/152: symlink de/d11/d23/d24/l31 0 2024-03-11T19:55:51.760 INFO:tasks.workunit.client.0.smithi107.stdout:0/153: truncate de/f17 911269 0 2024-03-11T19:55:51.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/154: mkdir de/d11/d23/d32 0 2024-03-11T19:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout:5/196: dwrite da/dd/f28 [4194304,4194304] 0 2024-03-11T19:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout:6/192: dwrite d10/d19/f31 [0,4194304] 0 2024-03-11T19:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout:6/193: truncate d10/d16/f1e 894183 0 2024-03-11T19:55:51.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/155: dwrite f3 [0,4194304] 0 2024-03-11T19:55:51.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/156: fsync f4 0 2024-03-11T19:55:51.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/194: write d10/d16/f1e [14415,6079] 0 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:0/155: dread f5 [0,4194304] 0 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/195: creat d10/d16/d2b/f43 x:0 0 0 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/157: write dd/de/f15 [1353315,40725] 0 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/158: chown dd/f21 2186 1 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/159: dread - dd/de/d1b/d20/f26 zero size 2024-03-11T19:55:51.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/160: write dd/d13/d18/f1a [4015588,8385] 0 2024-03-11T19:55:51.795 INFO:tasks.workunit.client.0.smithi107.stdout:9/134: dwrite d4/d9/f14 [0,4194304] 0 2024-03-11T19:55:51.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/197: dread da/dd/f13 [0,4194304] 0 2024-03-11T19:55:51.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/198: chown da/d31/c39 2591 1 2024-03-11T19:55:51.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/196: link f0 d10/d16/d24/f44 0 2024-03-11T19:55:51.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/161: link dd/de/d1b/f1d dd/de/d1b/d20/f29 0 2024-03-11T19:55:51.803 INFO:tasks.workunit.client.0.smithi107.stdout:9/135: unlink d4/d1d/d1f/c21 0 2024-03-11T19:55:51.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/199: truncate da/fb 775616 0 2024-03-11T19:55:51.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/136: mkdir d4/d1d/d1f/d30 0 2024-03-11T19:55:51.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/137: chown d4/d9 423 1 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/156: dread fd [4194304,4194304] 0 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:9/138: link d4/lc d4/d22/l31 0 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/157: unlink de/d11/l25 0 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:9/139: chown d4/d22/f2d 11250677 1 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:9/140: fdatasync d4/f11 0 2024-03-11T19:55:51.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/158: write de/f2f [297902,109974] 0 2024-03-11T19:55:51.817 INFO:tasks.workunit.client.0.smithi107.stdout:8/162: dread dd/d13/d18/f1c [0,4194304] 0 2024-03-11T19:55:51.817 INFO:tasks.workunit.client.0.smithi107.stdout:8/163: write dd/de/d1b/f1d [6607890,4531] 0 2024-03-11T19:55:51.852 INFO:tasks.workunit.client.0.smithi107.stdout:5/200: dwrite da/f1a [0,4194304] 0 2024-03-11T19:55:51.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/201: mknod da/d14/d19/c3b 0 2024-03-11T19:55:51.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/202: mknod da/d14/d18/d1e/c3c 0 2024-03-11T19:55:51.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/203: truncate da/dd/f28 9111493 0 2024-03-11T19:55:51.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/159: dwrite de/f16 [8388608,4194304] 0 2024-03-11T19:55:51.869 INFO:tasks.workunit.client.0.smithi107.stdout:0/160: mknod de/d11/d23/d24/c33 0 2024-03-11T19:55:51.869 INFO:tasks.workunit.client.0.smithi107.stdout:0/161: mkdir de/d11/d23/d32/d34 0 2024-03-11T19:55:51.872 INFO:tasks.workunit.client.0.smithi107.stdout:8/164: dread dd/d13/d18/f1c [0,4194304] 0 2024-03-11T19:55:51.872 INFO:tasks.workunit.client.0.smithi107.stdout:8/165: chown dd/d13 0 1 2024-03-11T19:55:51.876 INFO:tasks.workunit.client.0.smithi107.stdout:5/204: dread f5 [4194304,4194304] 0 2024-03-11T19:55:51.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/141: dwrite d4/d1d/d1f/f2f [0,4194304] 0 2024-03-11T19:55:51.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/142: getdents d4/d1d/d1f/d30 0 2024-03-11T19:55:51.887 INFO:tasks.workunit.client.0.smithi107.stdout:8/166: truncate dd/de/d1b/f1d 2506716 0 2024-03-11T19:55:51.888 INFO:tasks.workunit.client.0.smithi107.stdout:5/205: symlink da/dd/l3d 0 2024-03-11T19:55:51.893 INFO:tasks.workunit.client.0.smithi107.stdout:9/143: creat d4/d1d/d1f/d28/f32 x:0 0 0 2024-03-11T19:55:51.893 INFO:tasks.workunit.client.0.smithi107.stdout:9/144: chown d4/d9 21 1 2024-03-11T19:55:51.894 INFO:tasks.workunit.client.0.smithi107.stdout:9/145: symlink d4/d22/l33 0 2024-03-11T19:55:51.896 INFO:tasks.workunit.client.0.smithi107.stdout:5/206: unlink da/d14/d19/c2b 0 2024-03-11T19:55:51.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/207: mkdir da/d14/d19/d3e 0 2024-03-11T19:55:51.905 INFO:tasks.workunit.client.0.smithi107.stdout:0/162: dwrite de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:51.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/163: rmdir de/d11/d23/d24 39 2024-03-11T19:55:51.907 INFO:tasks.workunit.client.0.smithi107.stdout:0/164: creat de/d11/f35 x:0 0 0 2024-03-11T19:55:51.932 INFO:tasks.workunit.client.0.smithi107.stdout:8/167: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:51.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/186: sync 2024-03-11T19:55:51.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/208: dwrite da/fb [0,4194304] 0 2024-03-11T19:55:51.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/209: readlink l7 0 2024-03-11T19:55:51.944 INFO:tasks.workunit.client.0.smithi107.stdout:8/168: dread dd/d16/f1e [0,4194304] 0 2024-03-11T19:55:51.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/146: dwrite d4/d22/d17/f1b [0,4194304] 0 2024-03-11T19:55:51.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/147: dread - d4/f27 zero size 2024-03-11T19:55:51.952 INFO:tasks.workunit.client.0.smithi107.stdout:2/187: dread d1/d1d/f24 [0,4194304] 0 2024-03-11T19:55:51.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/169: dread f4 [0,4194304] 0 2024-03-11T19:55:51.966 INFO:tasks.workunit.client.0.smithi107.stdout:5/210: dread f5 [0,4194304] 0 2024-03-11T19:55:51.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/170: dread - dd/de/d1b/d20/f26 zero size 2024-03-11T19:55:51.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/165: dwrite de/d11/d23/d24/f2d [0,4194304] 0 2024-03-11T19:55:51.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/211: rmdir da/d14/d19/d3e 0 2024-03-11T19:55:51.969 INFO:tasks.workunit.client.0.smithi107.stdout:8/171: mknod dd/d16/c2a 0 2024-03-11T19:55:51.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/212: fsync da/dd/f2d 0 2024-03-11T19:55:51.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/172: creat dd/d13/d22/f2b x:0 0 0 2024-03-11T19:55:51.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/173: fsync dd/d13/d18/f1c 0 2024-03-11T19:55:51.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/148: read d4/d9/f14 [3866097,118267] 0 2024-03-11T19:55:51.974 INFO:tasks.workunit.client.0.smithi107.stdout:9/149: rmdir d4/d1d/d1f/d30 0 2024-03-11T19:55:51.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/150: rmdir d4/d1d/d1f 39 2024-03-11T19:55:51.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/151: creat d4/d1d/d1f/d28/d2c/f34 x:0 0 0 2024-03-11T19:55:51.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/152: mknod d4/d1d/d1f/c35 0 2024-03-11T19:55:51.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/213: dread da/dd/f11 [0,4194304] 0 2024-03-11T19:55:51.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/214: stat da/d14/d18/d1e/l2a 0 2024-03-11T19:55:51.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/215: dread - da/f32 zero size 2024-03-11T19:55:51.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/153: symlink d4/d1d/d1f/l36 0 2024-03-11T19:55:51.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/166: write de/d11/d23/d26/f29 [1746083,106723] 0 2024-03-11T19:55:51.992 INFO:tasks.workunit.client.0.smithi107.stdout:9/154: mkdir d4/d1c/d37 0 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/188: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/189: write d1/d1a/d1e/d26/f30 [315624,65331] 0 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:9/155: rename d4/f11 to d4/d1c/f38 0 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/167: rmdir de/d11/d23/d26 39 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:9/156: creat d4/d1d/d1f/f39 x:0 0 0 2024-03-11T19:55:51.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/168: creat de/d11/d23/d32/d34/f36 x:0 0 0 2024-03-11T19:55:52.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/157: unlink d4/d1d/d1f/d28/f32 0 2024-03-11T19:55:52.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/158: stat d4/d1d/d1f/c26 0 2024-03-11T19:55:52.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/159: mknod d4/d9/c3a 0 2024-03-11T19:55:52.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/160: creat d4/d1d/f3b x:0 0 0 2024-03-11T19:55:52.010 INFO:tasks.workunit.client.0.smithi107.stdout:9/161: getdents d4/d1c 0 2024-03-11T19:55:52.010 INFO:tasks.workunit.client.0.smithi107.stdout:9/162: chown f1 312561480 1 2024-03-11T19:55:52.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/163: symlink d4/d22/l3c 0 2024-03-11T19:55:52.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/164: write f1 [9059774,118748] 0 2024-03-11T19:55:52.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/165: rename d4/d1d to d4/d1d/d1f/d3d 22 2024-03-11T19:55:52.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/166: rename d4/d1d/c2e to d4/d1d/d1f/d28/d2c/c3e 0 2024-03-11T19:55:52.025 INFO:tasks.workunit.client.0.smithi107.stdout:2/190: dread d1/d1a/d1e/d26/f32 [4194304,4194304] 0 2024-03-11T19:55:52.025 INFO:tasks.workunit.client.0.smithi107.stdout:2/191: chown d1/l27 1380 1 2024-03-11T19:55:52.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/174: dwrite dd/d16/f19 [0,4194304] 0 2024-03-11T19:55:52.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/175: mkdir dd/de/d1b/d2c 0 2024-03-11T19:55:52.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/176: creat dd/de/d1b/d2c/f2d x:0 0 0 2024-03-11T19:55:52.030 INFO:tasks.workunit.client.0.smithi107.stdout:8/177: creat dd/f2e x:0 0 0 2024-03-11T19:55:52.030 INFO:tasks.workunit.client.0.smithi107.stdout:8/178: stat f4 0 2024-03-11T19:55:52.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/192: dread d1/d1a/f20 [0,4194304] 0 2024-03-11T19:55:52.032 INFO:tasks.workunit.client.0.smithi107.stdout:8/179: symlink dd/de/d24/l2f 0 2024-03-11T19:55:52.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/193: mknod d1/c3b 0 2024-03-11T19:55:52.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/194: truncate d1/fc 8121412 0 2024-03-11T19:55:52.036 INFO:tasks.workunit.client.0.smithi107.stdout:2/195: symlink d1/l3c 0 2024-03-11T19:55:52.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/166: sync 2024-03-11T19:55:52.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/167: dread - d6/d19/d1e/d27/d2c/f34 zero size 2024-03-11T19:55:52.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/168: fsync f4 0 2024-03-11T19:55:52.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/196: dread d1/d1a/f20 [0,4194304] 0 2024-03-11T19:55:52.046 INFO:tasks.workunit.client.0.smithi107.stdout:2/197: write d1/f10 [4111072,17973] 0 2024-03-11T19:55:52.046 INFO:tasks.workunit.client.0.smithi107.stdout:2/198: write d1/f36 [964920,111380] 0 2024-03-11T19:55:52.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/167: write d4/d1d/d1f/f2f [3469011,23050] 0 2024-03-11T19:55:52.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/216: dwrite da/f36 [4194304,4194304] 0 2024-03-11T19:55:52.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/169: dwrite de/f16 [8388608,4194304] 0 2024-03-11T19:55:52.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/169: dread d6/d19/d1e/f21 [0,4194304] 0 2024-03-11T19:55:52.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/170: write d6/d19/d1e/f21 [618939,5636] 0 2024-03-11T19:55:52.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/216: sync 2024-03-11T19:55:52.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/217: write d6/da/d16/f2e [1855042,88973] 0 2024-03-11T19:55:52.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/171: mkdir d6/d19/d3d 0 2024-03-11T19:55:52.050 INFO:tasks.workunit.client.0.smithi107.stdout:1/172: write d6/f14 [1027437,127987] 0 2024-03-11T19:55:52.050 INFO:tasks.workunit.client.0.smithi107.stdout:3/179: sync 2024-03-11T19:55:52.052 INFO:tasks.workunit.client.0.smithi107.stdout:9/168: rmdir d4/d1c 39 2024-03-11T19:55:52.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/170: mknod de/d11/d23/c37 0 2024-03-11T19:55:52.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/171: chown de/d11 289808534 1 2024-03-11T19:55:52.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/169: creat d4/d1d/d1f/d28/d2c/f3f x:0 0 0 2024-03-11T19:55:52.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/217: creat da/d14/d38/f3f x:0 0 0 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/218: rename d6/c2d to d6/da/d16/c3d 0 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/180: write dd/f21 [834182,101179] 0 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:4/192: sync 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:6/197: sync 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/218: creat da/dd/f40 x:0 0 0 2024-03-11T19:55:52.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/181: symlink dd/de/d1b/d2c/l30 0 2024-03-11T19:55:52.068 INFO:tasks.workunit.client.0.smithi107.stdout:4/193: rmdir d4/d7/db 39 2024-03-11T19:55:52.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/219: read d6/d21/f26 [353583,70641] 0 2024-03-11T19:55:52.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/198: symlink d10/d16/d2b/l45 0 2024-03-11T19:55:52.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/194: creat d4/d7/d1f/d2b/d24/f3e x:0 0 0 2024-03-11T19:55:52.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/173: dwrite d6/f1d [4194304,4194304] 0 2024-03-11T19:55:52.097 INFO:tasks.workunit.client.0.smithi107.stdout:1/174: truncate d6/fd 2824513 0 2024-03-11T19:55:52.097 INFO:tasks.workunit.client.0.smithi107.stdout:1/175: creat d6/d19/d1e/d27/f3e x:0 0 0 2024-03-11T19:55:52.098 INFO:tasks.workunit.client.0.smithi107.stdout:1/176: dread - d6/d19/d1e/d27/d2c/f30 zero size 2024-03-11T19:55:52.098 INFO:tasks.workunit.client.0.smithi107.stdout:1/177: fsync d6/d19/d1e/d27/d2c/f30 0 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/178: write d6/d19/d1e/f24 [371569,85576] 0 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/220: rename d6/d9/d13/d1b/l37 to d6/d21/l3e 0 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/179: chown d6/d28 150113974 1 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/221: chown d6/da/d16/f2c 6 1 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/180: write d6/f1b [76919,129268] 0 2024-03-11T19:55:52.103 INFO:tasks.workunit.client.0.smithi107.stdout:4/195: rmdir d4/d7/d2f/d3b 39 2024-03-11T19:55:52.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/199: getdents d10/d16/d2b 0 2024-03-11T19:55:52.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/222: rmdir d6/d9/d13 39 2024-03-11T19:55:52.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/181: rename d6/d19/d1e/f3c to d6/d28/f3f 0 2024-03-11T19:55:52.107 INFO:tasks.workunit.client.0.smithi107.stdout:7/223: getdents d6/d9 0 2024-03-11T19:55:52.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/182: truncate d6/d19/d1e/f21 170194 0 2024-03-11T19:55:52.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/183: readlink d6/l18 0 2024-03-11T19:55:52.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/184: write d6/d19/d1e/f24 [718252,59310] 0 2024-03-11T19:55:52.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/185: creat d6/d19/f40 x:0 0 0 2024-03-11T19:55:52.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/180: dwrite d1/d12/f2f [0,4194304] 0 2024-03-11T19:55:52.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/186: creat d6/d28/f41 x:0 0 0 2024-03-11T19:55:52.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/187: fsync d6/d28/f3f 0 2024-03-11T19:55:52.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/188: write f4 [1543758,2319] 0 2024-03-11T19:55:52.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/189: stat d6/la 0 2024-03-11T19:55:52.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/181: creat d1/d12/d1a/f37 x:0 0 0 2024-03-11T19:55:52.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/190: symlink d6/d19/d1e/d27/d2c/l42 0 2024-03-11T19:55:52.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/191: dread - d6/d19/d1e/d27/f3e zero size 2024-03-11T19:55:52.120 INFO:tasks.workunit.client.0.smithi107.stdout:8/182: write dd/d13/d18/f1a [184712,108363] 0 2024-03-11T19:55:52.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/172: dwrite de/f1e [0,4194304] 0 2024-03-11T19:55:52.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/173: write de/d11/f1a [9062,5013] 0 2024-03-11T19:55:52.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/174: read de/f17 [879228,93191] 0 2024-03-11T19:55:52.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/175: truncate de/d11/d23/f2e 117151 0 2024-03-11T19:55:52.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/176: chown de/d11/d23/d26 1 1 2024-03-11T19:55:52.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/199: dwrite d1/f10 [4194304,4194304] 0 2024-03-11T19:55:52.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/200: truncate d1/d1a/f2c 568681 0 2024-03-11T19:55:52.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/183: getdents dd 0 2024-03-11T19:55:52.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/177: symlink de/d11/l38 0 2024-03-11T19:55:52.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/224: dread d6/da/f1e [0,4194304] 0 2024-03-11T19:55:52.130 INFO:tasks.workunit.client.0.smithi107.stdout:8/184: mkdir dd/d31 0 2024-03-11T19:55:52.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/225: rename d6/f22 to d6/da/d16/f3f 0 2024-03-11T19:55:52.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/226: creat d6/d21/f40 x:0 0 0 2024-03-11T19:55:52.134 INFO:tasks.workunit.client.0.smithi107.stdout:8/185: write dd/de/d1b/d2c/f2d [118843,99984] 0 2024-03-11T19:55:52.136 INFO:tasks.workunit.client.0.smithi107.stdout:2/201: dread d1/d1a/f2f [0,4194304] 0 2024-03-11T19:55:52.137 INFO:tasks.workunit.client.0.smithi107.stdout:7/227: rename d6/f29 to d6/d21/f41 0 2024-03-11T19:55:52.137 INFO:tasks.workunit.client.0.smithi107.stdout:7/228: stat d6/d21/f41 0 2024-03-11T19:55:52.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/200: dwrite d10/d16/f2d [0,4194304] 0 2024-03-11T19:55:52.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/201: dread - d10/d16/d24/f27 zero size 2024-03-11T19:55:52.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/202: symlink d1/d1a/d1e/d26/l3d 0 2024-03-11T19:55:52.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/229: mknod d6/da/d16/c42 0 2024-03-11T19:55:52.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/230: readlink d6/d21/l3e 0 2024-03-11T19:55:52.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/231: creat d6/f43 x:0 0 0 2024-03-11T19:55:52.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/232: fsync d6/da/d16/f2e 0 2024-03-11T19:55:52.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/203: creat d1/d1d/f3e x:0 0 0 2024-03-11T19:55:52.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/204: chown d1/d1a/c1b 5161221 1 2024-03-11T19:55:52.142 INFO:tasks.workunit.client.0.smithi107.stdout:0/178: dread de/d11/f1a [0,4194304] 0 2024-03-11T19:55:52.142 INFO:tasks.workunit.client.0.smithi107.stdout:0/179: chown de/d11/d23/d32/d34 677558 1 2024-03-11T19:55:52.145 INFO:tasks.workunit.client.0.smithi107.stdout:8/186: mkdir dd/d13/d32 0 2024-03-11T19:55:52.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/192: dread d6/d28/f3f [0,4194304] 0 2024-03-11T19:55:52.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/193: readlink d6/d19/d1e/d27/l3b 0 2024-03-11T19:55:52.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/233: symlink d6/d21/l44 0 2024-03-11T19:55:52.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/194: write d6/d19/d1e/d27/f3e [532382,124462] 0 2024-03-11T19:55:52.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/195: chown d6/f14 36013 1 2024-03-11T19:55:52.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/205: link d1/c3b d1/d1d/c3f 0 2024-03-11T19:55:52.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/206: creat d1/f40 x:0 0 0 2024-03-11T19:55:52.153 INFO:tasks.workunit.client.0.smithi107.stdout:0/180: unlink l6 0 2024-03-11T19:55:52.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/207: write d1/f40 [207142,12966] 0 2024-03-11T19:55:52.157 INFO:tasks.workunit.client.0.smithi107.stdout:0/181: mkdir de/d11/d23/d32/d34/d39 0 2024-03-11T19:55:52.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/208: rename d1/d1a/d1e/d26/d33 to d1/d1a/d1e/d41 0 2024-03-11T19:55:52.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/234: dread d6/da/f1f [0,4194304] 0 2024-03-11T19:55:52.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/235: chown d6/da/d16/f3f 95851595 1 2024-03-11T19:55:52.164 INFO:tasks.workunit.client.0.smithi107.stdout:0/182: mkdir de/d11/d23/d32/d34/d39/d3a 0 2024-03-11T19:55:52.166 INFO:tasks.workunit.client.0.smithi107.stdout:7/236: rename d6/d21/f40 to d6/da/f45 0 2024-03-11T19:55:52.168 INFO:tasks.workunit.client.0.smithi107.stdout:4/196: dwrite d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:52.169 INFO:tasks.workunit.client.0.smithi107.stdout:8/187: rename f3 to dd/d16/f33 0 2024-03-11T19:55:52.171 INFO:tasks.workunit.client.0.smithi107.stdout:8/188: dread - dd/f2e zero size 2024-03-11T19:55:52.171 INFO:tasks.workunit.client.0.smithi107.stdout:8/189: stat dd/d13 0 2024-03-11T19:55:52.171 INFO:tasks.workunit.client.0.smithi107.stdout:8/190: chown c8 9 1 2024-03-11T19:55:52.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/191: creat dd/d13/f34 x:0 0 0 2024-03-11T19:55:52.173 INFO:tasks.workunit.client.0.smithi107.stdout:1/196: dread d6/f1d [0,4194304] 0 2024-03-11T19:55:52.174 INFO:tasks.workunit.client.0.smithi107.stdout:8/192: chown dd/de/f15 317949270 1 2024-03-11T19:55:52.177 INFO:tasks.workunit.client.0.smithi107.stdout:4/197: dread d4/d7/d1f/d2b/d19/f23 [4194304,4194304] 0 2024-03-11T19:55:52.185 INFO:tasks.workunit.client.0.smithi107.stdout:3/182: dwrite d1/d19/f32 [0,4194304] 0 2024-03-11T19:55:52.185 INFO:tasks.workunit.client.0.smithi107.stdout:4/198: link d4/d7/f22 d4/d7/d1f/d2b/d24/d26/d29/f3f 0 2024-03-11T19:55:52.185 INFO:tasks.workunit.client.0.smithi107.stdout:4/199: dread - d4/d7/d2f/d3b/f31 zero size 2024-03-11T19:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/209: dread d1/f10 [0,4194304] 0 2024-03-11T19:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/202: dwrite d10/d16/d24/d2e/f3c [8388608,4194304] 0 2024-03-11T19:55:52.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/200: symlink d4/d7/d1f/l40 0 2024-03-11T19:55:52.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/201: chown d4/d7/c35 930 1 2024-03-11T19:55:52.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/210: creat d1/d1a/d34/f42 x:0 0 0 2024-03-11T19:55:52.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/211: readlink d1/l17 0 2024-03-11T19:55:52.196 INFO:tasks.workunit.client.0.smithi107.stdout:0/183: read de/f16 [6770968,81302] 0 2024-03-11T19:55:52.196 INFO:tasks.workunit.client.0.smithi107.stdout:6/203: rename d10/d16/f3a to d10/d16/d24/d2e/f46 0 2024-03-11T19:55:52.200 INFO:tasks.workunit.client.0.smithi107.stdout:4/202: symlink d4/d7/d1f/d2b/l41 0 2024-03-11T19:55:52.200 INFO:tasks.workunit.client.0.smithi107.stdout:4/203: dread - d4/d7/d1f/d2b/d24/d26/d29/f32 zero size 2024-03-11T19:55:52.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/184: truncate de/d11/f1b 1195518 0 2024-03-11T19:55:52.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/197: dread d6/d19/d1e/d27/f3e [0,4194304] 0 2024-03-11T19:55:52.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/198: readlink d6/d19/l36 0 2024-03-11T19:55:52.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/199: read f4 [2300745,85594] 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/200: creat d6/d19/f43 x:0 0 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/212: rename d1/d1d/c31 to d1/c43 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/201: write d6/d19/d1e/d27/d2c/f30 [327672,80219] 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:0/185: symlink de/l3b 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:6/204: symlink d10/d16/d2b/l47 0 2024-03-11T19:55:52.203 INFO:tasks.workunit.client.0.smithi107.stdout:6/205: read - d10/d16/d1f/f42 zero size 2024-03-11T19:55:52.204 INFO:tasks.workunit.client.0.smithi107.stdout:2/213: symlink d1/d1a/d1e/d41/l44 0 2024-03-11T19:55:52.206 INFO:tasks.workunit.client.0.smithi107.stdout:0/186: rename de/d11/d23/f2e to de/d11/d23/d32/d34/f3c 0 2024-03-11T19:55:52.206 INFO:tasks.workunit.client.0.smithi107.stdout:0/187: creat de/f3d x:0 0 0 2024-03-11T19:55:52.210 INFO:tasks.workunit.client.0.smithi107.stdout:6/206: mknod d10/d19/d38/c48 0 2024-03-11T19:55:52.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/188: mknod de/d11/d23/d26/c3e 0 2024-03-11T19:55:52.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/189: write de/f18 [133336,70621] 0 2024-03-11T19:55:52.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/183: write d1/d10/d26/d17/f1f [2734166,85505] 0 2024-03-11T19:55:52.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/184: read - d1/d12/d1a/f37 zero size 2024-03-11T19:55:52.233 INFO:tasks.workunit.client.0.smithi107.stdout:8/193: dwrite dd/d16/f33 [4194304,4194304] 0 2024-03-11T19:55:52.234 INFO:tasks.workunit.client.0.smithi107.stdout:7/237: dwrite d6/d9/f8 [0,4194304] 0 2024-03-11T19:55:52.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/185: truncate d1/d10/d13/f2c 1348061 0 2024-03-11T19:55:52.235 INFO:tasks.workunit.client.0.smithi107.stdout:7/238: mknod d6/c46 0 2024-03-11T19:55:52.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/239: write d6/d9/f39 [995394,84427] 0 2024-03-11T19:55:52.237 INFO:tasks.workunit.client.0.smithi107.stdout:8/194: rmdir dd/d13/d18 39 2024-03-11T19:55:52.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/186: unlink d1/d10/d26/d17/f1f 0 2024-03-11T19:55:52.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/187: chown d1/d19/f32 192 1 2024-03-11T19:55:52.240 INFO:tasks.workunit.client.0.smithi107.stdout:7/240: mkdir d6/da/d47 0 2024-03-11T19:55:52.240 INFO:tasks.workunit.client.0.smithi107.stdout:7/241: truncate d6/d9/d13/d1b/f1c 3623183 0 2024-03-11T19:55:52.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/195: mknod dd/d13/d32/c35 0 2024-03-11T19:55:52.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/196: fsync f4 0 2024-03-11T19:55:52.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/197: read - dd/d13/f34 zero size 2024-03-11T19:55:52.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/188: creat d1/d10/d2a/f38 x:0 0 0 2024-03-11T19:55:52.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/189: write d1/d12/d1a/f37 [654823,71116] 0 2024-03-11T19:55:52.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/190: stat d1/d19/f32 0 2024-03-11T19:55:52.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/191: write d1/d10/d26/f30 [4125235,15926] 0 2024-03-11T19:55:52.243 INFO:tasks.workunit.client.0.smithi107.stdout:7/242: symlink d6/d9/d13/d1b/l48 0 2024-03-11T19:55:52.243 INFO:tasks.workunit.client.0.smithi107.stdout:7/243: read - d6/d21/f3a zero size 2024-03-11T19:55:52.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/198: rmdir dd/d31 0 2024-03-11T19:55:52.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/199: write dd/d13/d22/f2b [373150,115369] 0 2024-03-11T19:55:52.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/200: dread - dd/f2e zero size 2024-03-11T19:55:52.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/201: creat dd/d13/d22/f36 x:0 0 0 2024-03-11T19:55:52.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/202: dwrite d6/d19/d1e/d27/f38 [0,4194304] 0 2024-03-11T19:55:52.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/203: chown d6/d19/d1e/d27/d2c/d32/c37 399298 1 2024-03-11T19:55:52.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/202: creat dd/d13/d18/f37 x:0 0 0 2024-03-11T19:55:52.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/203: write dd/d13/d18/f37 [72828,114242] 0 2024-03-11T19:55:52.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/204: symlink dd/de/d1b/d2c/l38 0 2024-03-11T19:55:52.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/204: dread d6/f14 [0,4194304] 0 2024-03-11T19:55:52.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/219: sync 2024-03-11T19:55:52.267 INFO:tasks.workunit.client.0.smithi107.stdout:9/170: sync 2024-03-11T19:55:52.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/190: dwrite de/f3d [0,4194304] 0 2024-03-11T19:55:52.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/191: read - de/f21 zero size 2024-03-11T19:55:52.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/204: dwrite d4/d7/f22 [0,4194304] 0 2024-03-11T19:55:52.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/205: write d4/f6 [410388,64864] 0 2024-03-11T19:55:52.290 INFO:tasks.workunit.client.0.smithi107.stdout:6/207: dread d10/d16/f33 [0,4194304] 0 2024-03-11T19:55:52.290 INFO:tasks.workunit.client.0.smithi107.stdout:7/244: dwrite d6/d21/f41 [4194304,4194304] 0 2024-03-11T19:55:52.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/214: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:55:52.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/215: write d1/f40 [711294,29834] 0 2024-03-11T19:55:52.293 INFO:tasks.workunit.client.0.smithi107.stdout:7/245: truncate d6/f2f 1368620 0 2024-03-11T19:55:52.293 INFO:tasks.workunit.client.0.smithi107.stdout:7/246: readlink d6/d21/l44 0 2024-03-11T19:55:52.293 INFO:tasks.workunit.client.0.smithi107.stdout:2/216: mknod d1/d1d/c45 0 2024-03-11T19:55:52.294 INFO:tasks.workunit.client.0.smithi107.stdout:9/171: dread d4/d9/f14 [0,4194304] 0 2024-03-11T19:55:52.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/208: symlink d10/d16/d2b/d39/l49 0 2024-03-11T19:55:52.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/209: fdatasync d10/d16/d24/d2e/f2f 0 2024-03-11T19:55:52.295 INFO:tasks.workunit.client.0.smithi107.stdout:7/247: write d6/da/d16/f2b [21820,54682] 0 2024-03-11T19:55:52.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/217: rmdir d1/d38 39 2024-03-11T19:55:52.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/205: dread dd/de/f15 [0,4194304] 0 2024-03-11T19:55:52.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/206: truncate dd/d13/d18/f1c 4926985 0 2024-03-11T19:55:52.298 INFO:tasks.workunit.client.0.smithi107.stdout:9/172: link d4/d1d/d1f/f2f d4/d22/d17/f40 0 2024-03-11T19:55:52.299 INFO:tasks.workunit.client.0.smithi107.stdout:7/248: mkdir d6/da/d49 0 2024-03-11T19:55:52.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/218: mkdir d1/d1a/d1e/d46 0 2024-03-11T19:55:52.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/173: rename d4/f2a to d4/d22/f41 0 2024-03-11T19:55:52.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/174: fsync d4/f16 0 2024-03-11T19:55:52.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/249: creat d6/da/d47/f4a x:0 0 0 2024-03-11T19:55:52.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/250: chown d6/da/f1e 102 1 2024-03-11T19:55:52.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/219: creat d1/d1a/d1e/d26/f47 x:0 0 0 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/192: write de/f1e [4005636,46873] 0 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/193: truncate de/f21 985853 0 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/194: chown l8 118356365 1 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/195: write de/f2f [939982,124201] 0 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/192: dwrite d1/d10/d26/f30 [0,4194304] 0 2024-03-11T19:55:52.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/193: write d1/d10/d2a/f38 [769822,85911] 0 2024-03-11T19:55:52.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/194: mkdir d1/d39 0 2024-03-11T19:55:52.321 INFO:tasks.workunit.client.0.smithi107.stdout:3/195: symlink d1/d19/d2d/l3a 0 2024-03-11T19:55:52.337 INFO:tasks.workunit.client.0.smithi107.stdout:1/205: dwrite d6/f14 [0,4194304] 0 2024-03-11T19:55:52.337 INFO:tasks.workunit.client.0.smithi107.stdout:5/220: dread da/d14/f35 [0,4194304] 0 2024-03-11T19:55:52.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/206: symlink d6/l44 0 2024-03-11T19:55:52.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/207: chown d6/d19/l36 704972 1 2024-03-11T19:55:52.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/208: chown d6 1495 1 2024-03-11T19:55:52.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/209: creat d6/d19/d1e/d27/d2c/f45 x:0 0 0 2024-03-11T19:55:52.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/210: truncate d6/d19/d1e/f24 1609654 0 2024-03-11T19:55:52.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/221: link da/d14/l22 da/d14/d18/l41 0 2024-03-11T19:55:52.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/196: write d1/d19/f32 [3002430,105090] 0 2024-03-11T19:55:52.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/196: dwrite de/f3d [0,4194304] 0 2024-03-11T19:55:52.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/197: fsync de/f18 0 2024-03-11T19:55:52.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/251: dwrite d6/da/d47/f4a [0,4194304] 0 2024-03-11T19:55:52.350 INFO:tasks.workunit.client.0.smithi107.stdout:5/222: rename da/dd to da/d31/d42 0 2024-03-11T19:55:52.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/223: truncate da/d14/d19/f1d 753429 0 2024-03-11T19:55:52.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/224: truncate da/d14/d18/f27 172411 0 2024-03-11T19:55:52.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/225: mkdir da/d14/d43 0 2024-03-11T19:55:52.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/226: mkdir da/d14/d18/d44 0 2024-03-11T19:55:52.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/227: creat da/d14/f45 x:0 0 0 2024-03-11T19:55:52.361 INFO:tasks.workunit.client.0.smithi107.stdout:5/228: rmdir da/d14/d18/d1e 39 2024-03-11T19:55:52.361 INFO:tasks.workunit.client.0.smithi107.stdout:1/211: read d6/f15 [1599874,102140] 0 2024-03-11T19:55:52.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/252: dread d6/da/d16/f2b [0,4194304] 0 2024-03-11T19:55:52.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/253: getdents d6/da/d49 0 2024-03-11T19:55:52.362 INFO:tasks.workunit.client.0.smithi107.stdout:7/254: write d6/f43 [757626,52016] 0 2024-03-11T19:55:52.365 INFO:tasks.workunit.client.0.smithi107.stdout:7/255: creat d6/da/d49/f4b x:0 0 0 2024-03-11T19:55:52.365 INFO:tasks.workunit.client.0.smithi107.stdout:0/198: read fd [5753699,130025] 0 2024-03-11T19:55:52.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/256: rename d6/f43 to d6/d21/f4c 0 2024-03-11T19:55:52.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/199: link de/f2f de/f3f 0 2024-03-11T19:55:52.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/257: mkdir d6/da/d4d 0 2024-03-11T19:55:52.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/200: getdents de/d11/d23/d24 0 2024-03-11T19:55:52.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/201: write de/f13 [959032,53314] 0 2024-03-11T19:55:52.370 INFO:tasks.workunit.client.0.smithi107.stdout:0/202: read de/f21 [505852,66340] 0 2024-03-11T19:55:52.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/212: dread d6/d19/d1e/f26 [0,4194304] 0 2024-03-11T19:55:52.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/213: creat d6/d19/f46 x:0 0 0 2024-03-11T19:55:52.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/197: dwrite d1/f5 [0,4194304] 0 2024-03-11T19:55:52.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/198: write d1/d19/f32 [507569,29929] 0 2024-03-11T19:55:52.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/199: creat d1/d10/d13/f3b x:0 0 0 2024-03-11T19:55:52.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/229: dwrite da/d31/d42/f13 [0,4194304] 0 2024-03-11T19:55:52.412 INFO:tasks.workunit.client.0.smithi107.stdout:7/258: dread d6/da/d16/f3f [0,4194304] 0 2024-03-11T19:55:52.413 INFO:tasks.workunit.client.0.smithi107.stdout:7/259: truncate d6/d21/f41 4952705 0 2024-03-11T19:55:52.415 INFO:tasks.workunit.client.0.smithi107.stdout:7/260: mkdir d6/d9/d4e 0 2024-03-11T19:55:52.417 INFO:tasks.workunit.client.0.smithi107.stdout:7/261: creat d6/d9/f4f x:0 0 0 2024-03-11T19:55:52.421 INFO:tasks.workunit.client.0.smithi107.stdout:5/230: dread da/d31/d42/f11 [0,4194304] 0 2024-03-11T19:55:52.421 INFO:tasks.workunit.client.0.smithi107.stdout:5/231: readlink l7 0 2024-03-11T19:55:52.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/214: read d6/f1d [5282664,122355] 0 2024-03-11T19:55:52.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/215: rename d6/d19/d1e/f26 to d6/d19/d1e/d27/f47 0 2024-03-11T19:55:52.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/216: write d6/f1b [4356935,108228] 0 2024-03-11T19:55:52.427 INFO:tasks.workunit.client.0.smithi107.stdout:7/262: write d6/f17 [2984275,19140] 0 2024-03-11T19:55:52.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/217: creat d6/f48 x:0 0 0 2024-03-11T19:55:52.428 INFO:tasks.workunit.client.0.smithi107.stdout:7/263: rename d6/d9/d13/d1b/l48 to d6/d9/d4e/l50 0 2024-03-11T19:55:52.429 INFO:tasks.workunit.client.0.smithi107.stdout:0/203: dwrite de/d11/f35 [0,4194304] 0 2024-03-11T19:55:52.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/204: chown de/d11/d23/d32/d34/f3c 162 1 2024-03-11T19:55:52.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/205: getdents de/d11/d23/d24 0 2024-03-11T19:55:52.432 INFO:tasks.workunit.client.0.smithi107.stdout:1/218: rmdir d6/d19/d1e/d27 39 2024-03-11T19:55:52.433 INFO:tasks.workunit.client.0.smithi107.stdout:1/219: rmdir d6/d19/d1e/d27/d2c 39 2024-03-11T19:55:52.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/232: read da/f36 [4656868,55094] 0 2024-03-11T19:55:52.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/233: truncate da/d14/f3a 94233 0 2024-03-11T19:55:52.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/234: getdents da/d14 0 2024-03-11T19:55:52.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/264: dwrite d6/da/f1f [0,4194304] 0 2024-03-11T19:55:52.475 INFO:tasks.workunit.client.0.smithi107.stdout:7/265: chown d6/da/d49 1665 1 2024-03-11T19:55:52.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/206: dwrite de/f3f [0,4194304] 0 2024-03-11T19:55:52.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/207: creat de/f40 x:0 0 0 2024-03-11T19:55:52.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/208: mknod de/c41 0 2024-03-11T19:55:52.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/220: dwrite d6/d19/d1e/f21 [0,4194304] 0 2024-03-11T19:55:52.506 INFO:tasks.workunit.client.0.smithi107.stdout:5/235: dwrite da/f32 [0,4194304] 0 2024-03-11T19:55:52.506 INFO:tasks.workunit.client.0.smithi107.stdout:5/236: stat da/d31/c39 0 2024-03-11T19:55:52.506 INFO:tasks.workunit.client.0.smithi107.stdout:0/209: dread de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:52.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/237: mkdir da/d14/d38/d46 0 2024-03-11T19:55:52.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/210: creat de/d11/d23/d26/f42 x:0 0 0 2024-03-11T19:55:52.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/211: readlink l1 0 2024-03-11T19:55:52.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/212: truncate de/f13 1446364 0 2024-03-11T19:55:52.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/213: fdatasync de/f40 0 2024-03-11T19:55:52.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/266: dwrite d6/d9/d13/d1b/f1c [0,4194304] 0 2024-03-11T19:55:52.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/267: creat d6/d9/d13/f51 x:0 0 0 2024-03-11T19:55:52.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/238: rmdir da/d14/d38 39 2024-03-11T19:55:52.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/239: readlink da/d14/l2e 0 2024-03-11T19:55:52.522 INFO:tasks.workunit.client.0.smithi107.stdout:1/221: dread f4 [0,4194304] 0 2024-03-11T19:55:52.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/222: mknod d6/c49 0 2024-03-11T19:55:52.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/223: fdatasync d6/d19/d1e/f29 0 2024-03-11T19:55:52.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/268: read d6/f2f [213335,66622] 0 2024-03-11T19:55:52.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/269: chown d6/d9/d13/d1b/f1c 0 1 2024-03-11T19:55:52.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/270: chown d6/da/d16/c18 127331203 1 2024-03-11T19:55:52.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/214: write f5 [583726,118275] 0 2024-03-11T19:55:52.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/271: unlink d6/da/d16/f1a 0 2024-03-11T19:55:52.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/272: link d6/c46 d6/d9/d13/c52 0 2024-03-11T19:55:52.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/273: rename d6/da/f1e to d6/da/d4d/f53 0 2024-03-11T19:55:52.540 INFO:tasks.workunit.client.0.smithi107.stdout:7/274: rename d6/da/d4d/f53 to d6/da/d47/f54 0 2024-03-11T19:55:52.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/275: rename d6/d9/c1d to d6/d9/d13/d1b/c55 0 2024-03-11T19:55:52.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/240: dwrite da/d31/d42/f11 [0,4194304] 0 2024-03-11T19:55:52.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/224: dwrite d6/d28/f41 [0,4194304] 0 2024-03-11T19:55:52.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/225: dread - d6/f48 zero size 2024-03-11T19:55:52.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/226: mkdir d6/d4a 0 2024-03-11T19:55:52.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/227: write d6/d19/d1e/d27/f47 [1085934,74061] 0 2024-03-11T19:55:52.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/228: creat d6/d28/f4b x:0 0 0 2024-03-11T19:55:52.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/215: dwrite de/f18 [0,4194304] 0 2024-03-11T19:55:52.564 INFO:tasks.workunit.client.0.smithi107.stdout:9/175: sync 2024-03-11T19:55:52.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/176: dread - d4/d1d/f3b zero size 2024-03-11T19:55:52.565 INFO:tasks.workunit.client.0.smithi107.stdout:6/210: sync 2024-03-11T19:55:52.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/207: sync 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:8/208: fsync dd/de/d1b/d20/f29 0 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/206: sync 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:3/200: sync 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:2/220: sync 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:9/177: link d4/d22/f41 d4/d1d/d1f/d28/f42 0 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:9/178: fsync f1 0 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/211: symlink d10/d19/l4a 0 2024-03-11T19:55:52.573 INFO:tasks.workunit.client.0.smithi107.stdout:3/201: unlink d1/ld 0 2024-03-11T19:55:52.574 INFO:tasks.workunit.client.0.smithi107.stdout:4/207: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/c42 0 2024-03-11T19:55:52.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/212: symlink d10/d19/d38/l4b 0 2024-03-11T19:55:52.577 INFO:tasks.workunit.client.0.smithi107.stdout:4/208: link d4/d7/d1f/d2b/d24/d26/f38 d4/d7/d1f/d2b/d24/d26/f43 0 2024-03-11T19:55:52.577 INFO:tasks.workunit.client.0.smithi107.stdout:4/209: fsync d4/d7/d1f/d2b/d24/f3e 0 2024-03-11T19:55:52.579 INFO:tasks.workunit.client.0.smithi107.stdout:6/213: truncate fc 7671206 0 2024-03-11T19:55:52.580 INFO:tasks.workunit.client.0.smithi107.stdout:9/179: fsync d4/d1d/d1f/d28/f42 0 2024-03-11T19:55:52.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/210: creat d4/d7/d2f/d33/d3a/f44 x:0 0 0 2024-03-11T19:55:52.582 INFO:tasks.workunit.client.0.smithi107.stdout:5/241: write da/fb [2307976,95001] 0 2024-03-11T19:55:52.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/214: truncate fd 339080 0 2024-03-11T19:55:52.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/215: write d10/d19/f23 [490943,61655] 0 2024-03-11T19:55:52.584 INFO:tasks.workunit.client.0.smithi107.stdout:6/216: stat d10/f14 0 2024-03-11T19:55:52.584 INFO:tasks.workunit.client.0.smithi107.stdout:6/217: fsync f0 0 2024-03-11T19:55:52.584 INFO:tasks.workunit.client.0.smithi107.stdout:9/180: unlink d4/d22/l31 0 2024-03-11T19:55:52.584 INFO:tasks.workunit.client.0.smithi107.stdout:9/181: fdatasync d4/d22/d17/f1b 0 2024-03-11T19:55:52.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/242: mkdir da/d14/d43/d47 0 2024-03-11T19:55:52.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/182: rename d4/d1c/c23 to d4/c43 0 2024-03-11T19:55:52.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/183: write d4/d22/d17/f40 [5227682,127672] 0 2024-03-11T19:55:52.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/184: chown d4/d9/fa 14 1 2024-03-11T19:55:52.597 INFO:tasks.workunit.client.0.smithi107.stdout:9/185: getdents d4/d1d/d1f/d28/d2c 0 2024-03-11T19:55:52.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/216: dread de/f18 [0,4194304] 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:9/186: truncate d4/d1c/f38 1448052 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/211: write d4/d7/db/f21 [2136522,90988] 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/217: dread de/f17 [0,4194304] 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/212: mknod d4/d7/db/c45 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/213: write d4/d7/d2f/d33/d3a/f44 [613616,112277] 0 2024-03-11T19:55:52.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/218: creat de/d11/d23/d26/f43 x:0 0 0 2024-03-11T19:55:52.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/219: creat de/f44 x:0 0 0 2024-03-11T19:55:52.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/220: fsync de/f13 0 2024-03-11T19:55:52.608 INFO:tasks.workunit.client.0.smithi107.stdout:8/209: dwrite dd/d13/d18/f37 [0,4194304] 0 2024-03-11T19:55:52.608 INFO:tasks.workunit.client.0.smithi107.stdout:4/214: symlink d4/d7/d1f/d2b/d24/d26/d29/l46 0 2024-03-11T19:55:52.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/221: creat de/d11/f45 x:0 0 0 2024-03-11T19:55:52.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/222: chown de/f17 3337916 1 2024-03-11T19:55:52.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/210: unlink dd/d13/d22/f36 0 2024-03-11T19:55:52.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/211: chown dd/de/c11 66226751 1 2024-03-11T19:55:52.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/229: write d6/fd [1104020,116626] 0 2024-03-11T19:55:52.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/230: dread - d6/f16 zero size 2024-03-11T19:55:52.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/215: rename d4/d7/d2f/d3b/f31 to d4/d7/d2f/d3c/f47 0 2024-03-11T19:55:52.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/216: write d4/d7/d1f/d2b/d24/d26/d29/f3f [4446212,108040] 0 2024-03-11T19:55:52.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/217: write d4/d7/f8 [2052475,55098] 0 2024-03-11T19:55:52.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/218: stat d4/d7/d1f/d2b/d24/f3e 0 2024-03-11T19:55:52.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/202: dread d1/d12/d1a/f37 [0,4194304] 0 2024-03-11T19:55:52.619 INFO:tasks.workunit.client.0.smithi107.stdout:7/276: dwrite d6/f17 [0,4194304] 0 2024-03-11T19:55:52.629 INFO:tasks.workunit.client.0.smithi107.stdout:3/203: dread d1/d12/d1a/f37 [0,4194304] 0 2024-03-11T19:55:52.629 INFO:tasks.workunit.client.0.smithi107.stdout:3/204: fdatasync d1/d12/f2f 0 2024-03-11T19:55:52.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/243: write f4 [3912065,12196] 0 2024-03-11T19:55:52.631 INFO:tasks.workunit.client.0.smithi107.stdout:0/223: write de/f2f [924348,77386] 0 2024-03-11T19:55:52.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/277: mknod d6/da/d49/c56 0 2024-03-11T19:55:52.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/278: write d6/da/f45 [203737,72637] 0 2024-03-11T19:55:52.642 INFO:tasks.workunit.client.0.smithi107.stdout:3/205: rmdir d1/d39 0 2024-03-11T19:55:52.642 INFO:tasks.workunit.client.0.smithi107.stdout:3/206: write d1/fa [589330,22669] 0 2024-03-11T19:55:52.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/207: creat d1/d19/d2d/f3c x:0 0 0 2024-03-11T19:55:52.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/231: write d6/d19/d1e/f29 [3745853,4407] 0 2024-03-11T19:55:52.644 INFO:tasks.workunit.client.0.smithi107.stdout:4/219: truncate d4/d7/f8 2081084 0 2024-03-11T19:55:52.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/208: fdatasync d1/d12/f2f 0 2024-03-11T19:55:52.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/244: stat da/d14/d19/c29 0 2024-03-11T19:55:52.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/245: truncate da/d31/d42/f11 5098327 0 2024-03-11T19:55:52.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/246: rmdir da 39 2024-03-11T19:55:52.652 INFO:tasks.workunit.client.0.smithi107.stdout:1/232: unlink d6/d28/f2e 0 2024-03-11T19:55:52.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/212: dread dd/d13/d18/f1a [0,4194304] 0 2024-03-11T19:55:52.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/213: chown l9 1119 1 2024-03-11T19:55:52.654 INFO:tasks.workunit.client.0.smithi107.stdout:5/247: link da/d14/c17 da/d14/d38/d46/c48 0 2024-03-11T19:55:52.657 INFO:tasks.workunit.client.0.smithi107.stdout:8/214: symlink dd/d13/d32/l39 0 2024-03-11T19:55:52.658 INFO:tasks.workunit.client.0.smithi107.stdout:8/215: mknod dd/d13/d22/c3a 0 2024-03-11T19:55:52.658 INFO:tasks.workunit.client.0.smithi107.stdout:8/216: truncate dd/de/d1b/d2c/f2d 976267 0 2024-03-11T19:55:52.659 INFO:tasks.workunit.client.0.smithi107.stdout:8/217: creat dd/de/f3b x:0 0 0 2024-03-11T19:55:52.659 INFO:tasks.workunit.client.0.smithi107.stdout:8/218: chown dd/d13/d18/f1a 147 1 2024-03-11T19:55:52.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/219: mknod dd/de/d1b/d20/c3c 0 2024-03-11T19:55:52.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/220: creat dd/d13/f3d x:0 0 0 2024-03-11T19:55:52.664 INFO:tasks.workunit.client.0.smithi107.stdout:6/218: dwrite d10/d16/f1e [0,4194304] 0 2024-03-11T19:55:52.665 INFO:tasks.workunit.client.0.smithi107.stdout:6/219: write d10/d16/d1f/f42 [726281,121682] 0 2024-03-11T19:55:52.666 INFO:tasks.workunit.client.0.smithi107.stdout:6/220: creat d10/d3d/f4c x:0 0 0 2024-03-11T19:55:52.686 INFO:tasks.workunit.client.0.smithi107.stdout:9/187: dwrite d4/d22/f41 [0,4194304] 0 2024-03-11T19:55:52.687 INFO:tasks.workunit.client.0.smithi107.stdout:8/221: dread dd/d13/d18/f1c [0,4194304] 0 2024-03-11T19:55:52.689 INFO:tasks.workunit.client.0.smithi107.stdout:8/222: unlink dd/d13/d22/c3a 0 2024-03-11T19:55:52.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/224: dwrite de/f3d [0,4194304] 0 2024-03-11T19:55:52.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/223: rename dd/de/d24/l2f to dd/de/d1b/d2c/l3e 0 2024-03-11T19:55:52.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/224: chown dd/d13/d22 88668422 1 2024-03-11T19:55:52.696 INFO:tasks.workunit.client.0.smithi107.stdout:0/225: rename de/d11/d23/l2a to de/d11/d23/l46 0 2024-03-11T19:55:52.700 INFO:tasks.workunit.client.0.smithi107.stdout:8/225: mknod dd/de/c3f 0 2024-03-11T19:55:52.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/209: dread d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:52.702 INFO:tasks.workunit.client.0.smithi107.stdout:3/210: chown d1/d10/l15 164 1 2024-03-11T19:55:52.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/211: mknod d1/d12/c3d 0 2024-03-11T19:55:52.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/233: dwrite d6/d19/f43 [0,4194304] 0 2024-03-11T19:55:52.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/234: fsync d6/d19/f43 0 2024-03-11T19:55:52.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/235: chown d6/l23 574 1 2024-03-11T19:55:52.721 INFO:tasks.workunit.client.0.smithi107.stdout:7/279: dwrite d6/da/f1f [0,4194304] 0 2024-03-11T19:55:52.724 INFO:tasks.workunit.client.0.smithi107.stdout:1/236: rmdir d6/d19/d1e/d27 39 2024-03-11T19:55:52.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/237: rename d6/d19/f46 to d6/d19/d1e/d27/d2c/d32/f4c 0 2024-03-11T19:55:52.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/238: chown d6/d19/d1e/d27/d2c/f30 28019 1 2024-03-11T19:55:52.725 INFO:tasks.workunit.client.0.smithi107.stdout:7/280: mkdir d6/d9/d57 0 2024-03-11T19:55:52.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/239: creat d6/d19/d1e/f4d x:0 0 0 2024-03-11T19:55:52.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/248: dwrite da/f36 [4194304,4194304] 0 2024-03-11T19:55:52.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/240: rmdir d6/d4a 0 2024-03-11T19:55:52.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/188: dread d4/f12 [0,4194304] 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/221: dwrite d10/d19/f23 [0,4194304] 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:9/189: fdatasync d4/d22/d17/f19 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:9/190: chown d4/d9/fa 12522157 1 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/222: write d10/f14 [5183389,7774] 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/223: write d10/d16/d24/d2e/f2f [128189,17579] 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/224: fsync d10/d16/f17 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/241: rename d6/d19/d1e/d27/d2c/d32/f4c to d6/d19/d1e/f4e 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/225: truncate d10/d16/f1e 4977107 0 2024-03-11T19:55:52.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/242: fdatasync d6/d19/d1e/d27/d2c/f34 0 2024-03-11T19:55:52.745 INFO:tasks.workunit.client.0.smithi107.stdout:9/191: symlink d4/d1c/d37/l44 0 2024-03-11T19:55:52.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/226: rmdir d10/d16/d24/d2e 39 2024-03-11T19:55:52.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/227: chown d10/d16/d24/d2e/f46 48 1 2024-03-11T19:55:52.753 INFO:tasks.workunit.client.0.smithi107.stdout:8/226: dwrite dd/d16/f19 [4194304,4194304] 0 2024-03-11T19:55:52.753 INFO:tasks.workunit.client.0.smithi107.stdout:8/227: creat dd/d13/d18/f40 x:0 0 0 2024-03-11T19:55:52.755 INFO:tasks.workunit.client.0.smithi107.stdout:5/249: write da/f36 [1090081,7124] 0 2024-03-11T19:55:52.760 INFO:tasks.workunit.client.0.smithi107.stdout:5/250: creat da/d14/d18/d44/f49 x:0 0 0 2024-03-11T19:55:52.760 INFO:tasks.workunit.client.0.smithi107.stdout:5/251: chown da/d14 387175 1 2024-03-11T19:55:52.760 INFO:tasks.workunit.client.0.smithi107.stdout:5/252: stat da/d14/d19/c29 0 2024-03-11T19:55:52.762 INFO:tasks.workunit.client.0.smithi107.stdout:5/253: mknod da/d14/c4a 0 2024-03-11T19:55:52.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/254: mknod da/d14/d43/c4b 0 2024-03-11T19:55:52.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/226: dwrite de/f16 [0,4194304] 0 2024-03-11T19:55:52.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/227: chown de/d11/c22 7682 1 2024-03-11T19:55:52.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/228: truncate de/f21 1874662 0 2024-03-11T19:55:52.766 INFO:tasks.workunit.client.0.smithi107.stdout:5/255: creat da/d14/d18/f4c x:0 0 0 2024-03-11T19:55:52.766 INFO:tasks.workunit.client.0.smithi107.stdout:5/256: write da/d31/d42/f23 [1758064,35264] 0 2024-03-11T19:55:52.766 INFO:tasks.workunit.client.0.smithi107.stdout:5/257: fdatasync da/fb 0 2024-03-11T19:55:52.766 INFO:tasks.workunit.client.0.smithi107.stdout:5/258: rename da/d14 to da/d14/d18/d4d 22 2024-03-11T19:55:52.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/229: link de/d11/d23/d24/c27 de/d11/d23/d32/d34/c47 0 2024-03-11T19:55:52.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/230: dread - de/d11/f2c zero size 2024-03-11T19:55:52.771 INFO:tasks.workunit.client.0.smithi107.stdout:5/259: symlink da/d31/d42/l4e 0 2024-03-11T19:55:52.772 INFO:tasks.workunit.client.0.smithi107.stdout:5/260: creat da/d31/f4f x:0 0 0 2024-03-11T19:55:52.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/261: mknod da/d14/d18/d44/c50 0 2024-03-11T19:55:52.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/262: read - da/d31/f4f zero size 2024-03-11T19:55:52.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/263: chown da/d14/d38/f3f 5998 1 2024-03-11T19:55:52.778 INFO:tasks.workunit.client.0.smithi107.stdout:5/264: rename da/d31/d42/f2d to da/d14/d18/f51 0 2024-03-11T19:55:52.778 INFO:tasks.workunit.client.0.smithi107.stdout:5/265: rename da/d14 to da/d14/d43/d47/d52 22 2024-03-11T19:55:52.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/266: symlink da/d14/d38/d46/l53 0 2024-03-11T19:55:52.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/228: dread dd/d16/f19 [4194304,4194304] 0 2024-03-11T19:55:52.783 INFO:tasks.workunit.client.0.smithi107.stdout:8/229: mknod dd/d13/d22/c41 0 2024-03-11T19:55:52.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/231: dread f3 [0,4194304] 0 2024-03-11T19:55:52.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/228: dwrite f8 [0,4194304] 0 2024-03-11T19:55:52.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/229: readlink d10/d19/l35 0 2024-03-11T19:55:52.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/230: mknod d10/c4d 0 2024-03-11T19:55:52.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/231: rmdir d10/d16/d2b/d39 39 2024-03-11T19:55:52.793 INFO:tasks.workunit.client.0.smithi107.stdout:3/212: dwrite d1/d12/d1a/f37 [0,4194304] 0 2024-03-11T19:55:52.794 INFO:tasks.workunit.client.0.smithi107.stdout:8/230: getdents dd/d13/d22 0 2024-03-11T19:55:52.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/281: dread d6/da/d16/f36 [0,4194304] 0 2024-03-11T19:55:52.798 INFO:tasks.workunit.client.0.smithi107.stdout:8/231: creat dd/d13/d22/f42 x:0 0 0 2024-03-11T19:55:52.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/213: mkdir d1/d10/d26/d3e 0 2024-03-11T19:55:52.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/232: fdatasync dd/de/d1b/f1d 0 2024-03-11T19:55:52.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/221: sync 2024-03-11T19:55:52.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/222: stat d1/l2 0 2024-03-11T19:55:52.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/223: stat d1/d1a/d1e/d26 0 2024-03-11T19:55:52.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/224: getdents d1 0 2024-03-11T19:55:52.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/225: truncate d1/d1a/f20 12693 0 2024-03-11T19:55:52.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/226: write d1/d1d/f24 [751545,67526] 0 2024-03-11T19:55:52.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/267: read da/f32 [3949881,58328] 0 2024-03-11T19:55:52.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/282: write d6/da/d47/f54 [2355168,77786] 0 2024-03-11T19:55:52.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/232: dread de/f3d [0,4194304] 0 2024-03-11T19:55:52.818 INFO:tasks.workunit.client.0.smithi107.stdout:9/192: dwrite d4/d9/fa [0,4194304] 0 2024-03-11T19:55:52.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/233: creat de/d11/d23/d26/f48 x:0 0 0 2024-03-11T19:55:52.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/268: mkdir da/d14/d19/d54 0 2024-03-11T19:55:52.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/269: write da/d31/d42/f23 [112483,104168] 0 2024-03-11T19:55:52.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/193: symlink d4/d1d/d1f/d28/d2c/l45 0 2024-03-11T19:55:52.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/270: truncate da/d31/d42/f28 5848840 0 2024-03-11T19:55:52.825 INFO:tasks.workunit.client.0.smithi107.stdout:5/271: rmdir da/d14/d18 39 2024-03-11T19:55:52.826 INFO:tasks.workunit.client.0.smithi107.stdout:5/272: write da/d14/d38/f3f [868097,106900] 0 2024-03-11T19:55:52.850 INFO:tasks.workunit.client.0.smithi107.stdout:8/233: dwrite f4 [4194304,4194304] 0 2024-03-11T19:55:52.851 INFO:tasks.workunit.client.0.smithi107.stdout:7/283: dread d6/da/d47/f4a [0,4194304] 0 2024-03-11T19:55:52.853 INFO:tasks.workunit.client.0.smithi107.stdout:7/284: rename d6/d9/f39 to d6/da/d47/f58 0 2024-03-11T19:55:52.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/214: dwrite d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:52.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/215: chown d1/d10/c29 19 1 2024-03-11T19:55:52.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/216: readlink d1/d10/l1c 0 2024-03-11T19:55:52.858 INFO:tasks.workunit.client.0.smithi107.stdout:7/285: creat d6/d9/d4e/f59 x:0 0 0 2024-03-11T19:55:52.858 INFO:tasks.workunit.client.0.smithi107.stdout:7/286: fdatasync d6/da/d49/f4b 0 2024-03-11T19:55:52.858 INFO:tasks.workunit.client.0.smithi107.stdout:6/232: dwrite d10/d16/f20 [0,4194304] 0 2024-03-11T19:55:52.859 INFO:tasks.workunit.client.0.smithi107.stdout:6/233: chown d10/d3d 118207304 1 2024-03-11T19:55:52.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/217: truncate d1/d19/f32 1089533 0 2024-03-11T19:55:52.862 INFO:tasks.workunit.client.0.smithi107.stdout:6/234: creat d10/d19/d3b/f4e x:0 0 0 2024-03-11T19:55:52.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/218: rename d1/d12/c3d to d1/d19/c3f 0 2024-03-11T19:55:52.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/219: fsync d1/d10/d26/f30 0 2024-03-11T19:55:52.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/235: getdents d10/d16/d2b 0 2024-03-11T19:55:52.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/236: rmdir d10/d16/d2b/d39 39 2024-03-11T19:55:52.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/234: write de/f18 [2181780,58230] 0 2024-03-11T19:55:52.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/235: stat de/d11/d23/d24/l31 0 2024-03-11T19:55:52.879 INFO:tasks.workunit.client.0.smithi107.stdout:9/194: dwrite d4/fe [0,4194304] 0 2024-03-11T19:55:52.893 INFO:tasks.workunit.client.0.smithi107.stdout:8/234: write dd/d13/d18/f37 [1490702,60951] 0 2024-03-11T19:55:52.893 INFO:tasks.workunit.client.0.smithi107.stdout:8/235: chown dd/d13/f3d 2 1 2024-03-11T19:55:52.894 INFO:tasks.workunit.client.0.smithi107.stdout:3/220: dread d1/d10/d21/f27 [0,4194304] 0 2024-03-11T19:55:52.894 INFO:tasks.workunit.client.0.smithi107.stdout:2/227: dwrite d1/d1d/f3e [0,4194304] 0 2024-03-11T19:55:52.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/236: mknod de/d11/d23/d26/c49 0 2024-03-11T19:55:52.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/195: creat d4/d1d/d1f/d28/d2c/f46 x:0 0 0 2024-03-11T19:55:52.898 INFO:tasks.workunit.client.0.smithi107.stdout:6/237: link d10/d16/d2b/l41 d10/d19/d38/d40/l4f 0 2024-03-11T19:55:52.898 INFO:tasks.workunit.client.0.smithi107.stdout:6/238: readlink d10/d19/l35 0 2024-03-11T19:55:52.898 INFO:tasks.workunit.client.0.smithi107.stdout:3/221: truncate d1/d10/d13/f2c 616005 0 2024-03-11T19:55:52.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/239: symlink d10/d3d/l50 0 2024-03-11T19:55:52.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/196: mkdir d4/d1c/d47 0 2024-03-11T19:55:52.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/228: dread d1/d1d/f24 [0,4194304] 0 2024-03-11T19:55:52.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/229: creat d1/d1a/d1e/d26/f48 x:0 0 0 2024-03-11T19:55:52.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/240: unlink d10/d16/f17 0 2024-03-11T19:55:52.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/273: dwrite da/d31/d42/f1f [0,4194304] 0 2024-03-11T19:55:52.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/274: fdatasync da/d14/d18/f2c 0 2024-03-11T19:55:52.910 INFO:tasks.workunit.client.0.smithi107.stdout:5/275: write da/d14/d18/f27 [11421,113468] 0 2024-03-11T19:55:52.910 INFO:tasks.workunit.client.0.smithi107.stdout:5/276: fsync da/d14/f3a 0 2024-03-11T19:55:52.910 INFO:tasks.workunit.client.0.smithi107.stdout:5/277: read da/d14/d18/f51 [111074,56536] 0 2024-03-11T19:55:52.914 INFO:tasks.workunit.client.0.smithi107.stdout:2/230: creat d1/f49 x:0 0 0 2024-03-11T19:55:52.915 INFO:tasks.workunit.client.0.smithi107.stdout:8/236: dread dd/d13/d18/f1a [4194304,4194304] 0 2024-03-11T19:55:52.915 INFO:tasks.workunit.client.0.smithi107.stdout:8/237: fdatasync dd/de/f3b 0 2024-03-11T19:55:52.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/197: rmdir d4 39 2024-03-11T19:55:52.919 INFO:tasks.workunit.client.0.smithi107.stdout:2/231: rmdir d1/d1a/d1e/d41 39 2024-03-11T19:55:52.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/222: dread d1/d10/d26/f30 [0,4194304] 0 2024-03-11T19:55:52.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/223: readlink d1/d10/l1c 0 2024-03-11T19:55:52.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/224: truncate d1/d10/d13/f3b 622642 0 2024-03-11T19:55:52.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/225: fsync d1/d10/d13/f3b 0 2024-03-11T19:55:52.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/226: write d1/fa [376308,17208] 0 2024-03-11T19:55:52.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/241: mkdir d10/d16/d1f/d51 0 2024-03-11T19:55:52.924 INFO:tasks.workunit.client.0.smithi107.stdout:2/232: rename d1/d1a/d1e/d26/c29 to d1/d1a/c4a 0 2024-03-11T19:55:52.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/198: mknod d4/d1c/d47/c48 0 2024-03-11T19:55:52.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/199: write d4/d1d/d1f/d28/d2c/f34 [458470,88422] 0 2024-03-11T19:55:52.928 INFO:tasks.workunit.client.0.smithi107.stdout:2/233: symlink d1/d38/l4b 0 2024-03-11T19:55:52.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/242: rename d10/d16/d2b/l30 to d10/d19/l52 0 2024-03-11T19:55:52.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/237: dwrite de/f17 [0,4194304] 0 2024-03-11T19:55:52.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/200: link d4/f27 d4/d1d/d1f/d28/d2c/f49 0 2024-03-11T19:55:52.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/234: mknod d1/d21/c4c 0 2024-03-11T19:55:52.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/238: rename de/d11/f1a to de/d11/d23/d24/f4a 0 2024-03-11T19:55:52.935 INFO:tasks.workunit.client.0.smithi107.stdout:9/201: symlink d4/d1c/l4a 0 2024-03-11T19:55:52.937 INFO:tasks.workunit.client.0.smithi107.stdout:9/202: truncate f1 4420018 0 2024-03-11T19:55:52.937 INFO:tasks.workunit.client.0.smithi107.stdout:5/278: dread da/f1a [4194304,4194304] 0 2024-03-11T19:55:52.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/239: mknod de/d11/c4b 0 2024-03-11T19:55:52.939 INFO:tasks.workunit.client.0.smithi107.stdout:9/203: rmdir d4/d1d/d1f/d28/d2c 39 2024-03-11T19:55:52.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/279: getdents da/d14/d38/d46 0 2024-03-11T19:55:52.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/204: mkdir d4/d1d/d4b 0 2024-03-11T19:55:52.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/205: fsync d4/d1d/d1f/d28/d2c/f46 0 2024-03-11T19:55:52.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/235: dread d1/d1a/f2c [0,4194304] 0 2024-03-11T19:55:52.942 INFO:tasks.workunit.client.0.smithi107.stdout:3/227: dread d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:52.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/238: dread dd/d13/d18/f37 [0,4194304] 0 2024-03-11T19:55:52.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/280: rename da/d14/d43/c4b to da/d14/d18/c55 0 2024-03-11T19:55:52.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/206: readlink l3 0 2024-03-11T19:55:52.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/220: sync 2024-03-11T19:55:52.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/243: sync 2024-03-11T19:55:52.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/221: write d4/d7/d1f/d2b/d19/f2a [234991,79931] 0 2024-03-11T19:55:52.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/228: rmdir d1/d10/d26/d3e 0 2024-03-11T19:55:52.946 INFO:tasks.workunit.client.0.smithi107.stdout:5/281: creat da/d14/d38/f56 x:0 0 0 2024-03-11T19:55:52.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/229: write d1/d10/d26/f30 [5778929,124578] 0 2024-03-11T19:55:52.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/230: chown d1/d19/c3f 28 1 2024-03-11T19:55:52.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/222: getdents d4/d7/db 0 2024-03-11T19:55:52.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/223: chown d4/d7/d1f/d2b/d19/c1e 1437932 1 2024-03-11T19:55:52.954 INFO:tasks.workunit.client.0.smithi107.stdout:5/282: creat da/d14/d19/d54/f57 x:0 0 0 2024-03-11T19:55:52.958 INFO:tasks.workunit.client.0.smithi107.stdout:6/243: dread fc [4194304,4194304] 0 2024-03-11T19:55:52.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/244: creat d10/f53 x:0 0 0 2024-03-11T19:55:52.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/239: dread dd/d16/f1e [0,4194304] 0 2024-03-11T19:55:52.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/240: creat dd/de/d1b/d20/f43 x:0 0 0 2024-03-11T19:55:52.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/241: write dd/d13/d18/f37 [761615,74755] 0 2024-03-11T19:55:52.967 INFO:tasks.workunit.client.0.smithi107.stdout:0/240: write de/f3d [1565913,90443] 0 2024-03-11T19:55:52.967 INFO:tasks.workunit.client.0.smithi107.stdout:0/241: chown de/f44 57 1 2024-03-11T19:55:52.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/245: write d10/d16/d1f/f2a [5039,80707] 0 2024-03-11T19:55:52.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/207: dread d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:52.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/208: write d4/d9/f14 [713289,79486] 0 2024-03-11T19:55:52.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/244: write d6/d19/f43 [3447697,36333] 0 2024-03-11T19:55:52.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/245: stat d6/d19 0 2024-03-11T19:55:52.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/246: chown d6/d19/d1e/d27 140 1 2024-03-11T19:55:52.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/247: chown d6/l23 81633 1 2024-03-11T19:55:52.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/209: rename d4/c43 to d4/d1d/d1f/d28/c4c 0 2024-03-11T19:55:52.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/210: truncate d4/d1d/d1f/d28/d2c/f46 608821 0 2024-03-11T19:55:52.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/242: dread de/f13 [0,4194304] 0 2024-03-11T19:55:52.987 INFO:tasks.workunit.client.0.smithi107.stdout:9/211: mkdir d4/d9/d4d 0 2024-03-11T19:55:52.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/212: creat d4/d1d/f4e x:0 0 0 2024-03-11T19:55:52.996 INFO:tasks.workunit.client.0.smithi107.stdout:0/243: rename de/f1e to de/d11/f4c 0 2024-03-11T19:55:52.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/244: fdatasync de/d11/f1b 0 2024-03-11T19:55:52.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/213: symlink d4/d9/l4f 0 2024-03-11T19:55:53.000 INFO:tasks.workunit.client.0.smithi107.stdout:9/214: creat d4/d1c/d37/f50 x:0 0 0 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/215: mknod d4/d22/c51 0 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/216: readlink d4/d22/l3c 0 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/217: chown d4/d1c/d37/f50 161091 1 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/218: creat d4/d1d/f52 x:0 0 0 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/219: chown d4/d1c/d47 307882 1 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/220: chown d4/d1c/l4a 10148 1 2024-03-11T19:55:53.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/221: truncate d4/f12 4824303 0 2024-03-11T19:55:53.006 INFO:tasks.workunit.client.0.smithi107.stdout:0/245: rename de/d11/d23/d32/d34/f3c to de/d11/f4d 0 2024-03-11T19:55:53.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:52 smithi122 ceph-mon[36397]: pgmap v76: 129 pgs: 129 active+clean; 387 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 41 MiB/s wr, 336 op/s 2024-03-11T19:55:53.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/222: rename d4/d22/f41 to d4/f53 0 2024-03-11T19:55:53.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/223: truncate d4/d1d/f4e 436436 0 2024-03-11T19:55:53.012 INFO:tasks.workunit.client.0.smithi107.stdout:2/236: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:53.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/224: rename d4/d1d/f4e to d4/f54 0 2024-03-11T19:55:53.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/225: write d4/d1c/d37/f50 [460452,51480] 0 2024-03-11T19:55:53.017 INFO:tasks.workunit.client.0.smithi107.stdout:2/237: rmdir d1/d1a/d1e/d46 0 2024-03-11T19:55:53.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/238: getdents d1/d1a/d1e/d26 0 2024-03-11T19:55:53.022 INFO:tasks.workunit.client.0.smithi107.stdout:5/283: dwrite da/d14/d18/f2c [4194304,4194304] 0 2024-03-11T19:55:53.022 INFO:tasks.workunit.client.0.smithi107.stdout:5/284: chown da/d14/d19/c29 0 1 2024-03-11T19:55:53.031 INFO:tasks.workunit.client.0.smithi107.stdout:4/224: dwrite d4/d7/d1f/d2b/d24/d26/d29/f32 [0,4194304] 0 2024-03-11T19:55:53.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:52 smithi145 ceph-mon[36730]: pgmap v76: 129 pgs: 129 active+clean; 387 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 41 MiB/s wr, 336 op/s 2024-03-11T19:55:53.041 INFO:tasks.workunit.client.0.smithi107.stdout:8/242: dwrite dd/de/f3b [0,4194304] 0 2024-03-11T19:55:53.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:52 smithi107 ceph-mon[31712]: pgmap v76: 129 pgs: 129 active+clean; 387 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 41 MiB/s wr, 336 op/s 2024-03-11T19:55:53.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/248: dwrite d6/d19/d1e/d27/f3e [0,4194304] 0 2024-03-11T19:55:53.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/249: fdatasync d6/f15 0 2024-03-11T19:55:53.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/250: write d6/f15 [881871,119287] 0 2024-03-11T19:55:53.048 INFO:tasks.workunit.client.0.smithi107.stdout:7/287: sync 2024-03-11T19:55:53.048 INFO:tasks.workunit.client.0.smithi107.stdout:7/288: write d6/d9/d13/d1b/f1c [4595719,67630] 0 2024-03-11T19:55:53.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/289: rename d6/d21/f2a to d6/d9/d57/f5a 0 2024-03-11T19:55:53.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/290: readlink d6/d9/l34 0 2024-03-11T19:55:53.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/251: dread d6/fd [0,4194304] 0 2024-03-11T19:55:53.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/252: read d6/f1b [2525486,102591] 0 2024-03-11T19:55:53.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/253: write d6/d19/d1e/d27/d2c/f30 [1427046,20920] 0 2024-03-11T19:55:53.070 INFO:tasks.workunit.client.0.smithi107.stdout:1/254: read - d6/d19/d1e/d27/d2c/f45 zero size 2024-03-11T19:55:53.070 INFO:tasks.workunit.client.0.smithi107.stdout:1/255: symlink d6/d19/d1e/l4f 0 2024-03-11T19:55:53.072 INFO:tasks.workunit.client.0.smithi107.stdout:1/256: mknod d6/d19/d1e/d27/d2c/d32/c50 0 2024-03-11T19:55:53.072 INFO:tasks.workunit.client.0.smithi107.stdout:1/257: write f4 [120678,125986] 0 2024-03-11T19:55:53.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/258: getdents d6/d28 0 2024-03-11T19:55:53.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/246: dwrite d10/d16/f1c [0,4194304] 0 2024-03-11T19:55:53.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/239: dwrite d1/f49 [0,4194304] 0 2024-03-11T19:55:53.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/247: mkdir d10/d3d/d54 0 2024-03-11T19:55:53.080 INFO:tasks.workunit.client.0.smithi107.stdout:6/248: unlink d10/d16/d24/d2e/f3c 0 2024-03-11T19:55:53.083 INFO:tasks.workunit.client.0.smithi107.stdout:6/249: rename d10/d16/d2b/f3e to d10/d16/d24/d2e/f55 0 2024-03-11T19:55:53.088 INFO:tasks.workunit.client.0.smithi107.stdout:6/250: mknod d10/c56 0 2024-03-11T19:55:53.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/226: dwrite d4/d1d/d1f/d28/d2c/f34 [0,4194304] 0 2024-03-11T19:55:53.100 INFO:tasks.workunit.client.0.smithi107.stdout:3/231: sync 2024-03-11T19:55:53.100 INFO:tasks.workunit.client.0.smithi107.stdout:3/232: fdatasync d1/d10/d26/f25 0 2024-03-11T19:55:53.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/291: dwrite d6/d21/f3a [0,4194304] 0 2024-03-11T19:55:53.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/259: dread d6/f15 [0,4194304] 0 2024-03-11T19:55:53.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/260: truncate d6/d19/d1e/d27/d2c/f30 2224942 0 2024-03-11T19:55:53.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/233: rename d1/d19/c1e to d1/d12/d1a/c40 0 2024-03-11T19:55:53.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/234: chown d1/d10/d26 714 1 2024-03-11T19:55:53.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/261: symlink d6/d19/d1e/d27/d2c/l51 0 2024-03-11T19:55:53.105 INFO:tasks.workunit.client.0.smithi107.stdout:1/262: stat d6/f16 0 2024-03-11T19:55:53.105 INFO:tasks.workunit.client.0.smithi107.stdout:1/263: write f4 [3238074,117171] 0 2024-03-11T19:55:53.105 INFO:tasks.workunit.client.0.smithi107.stdout:3/235: truncate d1/d12/d1a/f37 123399 0 2024-03-11T19:55:53.106 INFO:tasks.workunit.client.0.smithi107.stdout:3/236: rename d1/d10/d21/f27 to d1/d12/d1a/f41 0 2024-03-11T19:55:53.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/243: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:53.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/244: stat dd/d16/c2a 0 2024-03-11T19:55:53.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/237: rmdir d1/d12/d1a 39 2024-03-11T19:55:53.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/238: chown d1/d10/d2a/f38 63 1 2024-03-11T19:55:53.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/239: readlink d1/d10/d13/l34 0 2024-03-11T19:55:53.113 INFO:tasks.workunit.client.0.smithi107.stdout:3/240: symlink d1/d12/d1a/l42 0 2024-03-11T19:55:53.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/241: mkdir d1/d10/d21/d43 0 2024-03-11T19:55:53.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/242: creat d1/d19/f44 x:0 0 0 2024-03-11T19:55:53.117 INFO:tasks.workunit.client.0.smithi107.stdout:3/243: mkdir d1/d45 0 2024-03-11T19:55:53.117 INFO:tasks.workunit.client.0.smithi107.stdout:3/244: stat d1/d19 0 2024-03-11T19:55:53.118 INFO:tasks.workunit.client.0.smithi107.stdout:3/245: link d1/d12/d1a/c2e d1/d10/d26/d17/c46 0 2024-03-11T19:55:53.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/227: dread d4/f12 [0,4194304] 0 2024-03-11T19:55:53.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/228: chown d4/d22/l33 31817 1 2024-03-11T19:55:53.126 INFO:tasks.workunit.client.0.smithi107.stdout:6/251: write d10/f13 [2251728,101747] 0 2024-03-11T19:55:53.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/240: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:53.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/229: rename d4/d22/l3c to d4/d1d/d1f/d28/l55 0 2024-03-11T19:55:53.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/230: mkdir d4/d1d/d1f/d28/d2c/d56 0 2024-03-11T19:55:53.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/231: rmdir d4/d1d/d1f/d28/d2c 39 2024-03-11T19:55:53.130 INFO:tasks.workunit.client.0.smithi107.stdout:9/232: mknod d4/d1d/d1f/d28/c57 0 2024-03-11T19:55:53.132 INFO:tasks.workunit.client.0.smithi107.stdout:9/233: link d4/d1d/f52 d4/f58 0 2024-03-11T19:55:53.147 INFO:tasks.workunit.client.0.smithi107.stdout:3/246: dread d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:53.148 INFO:tasks.workunit.client.0.smithi107.stdout:7/292: dread d6/da/d16/f36 [0,4194304] 0 2024-03-11T19:55:53.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/234: creat d4/d1d/d4b/f59 x:0 0 0 2024-03-11T19:55:53.151 INFO:tasks.workunit.client.0.smithi107.stdout:3/247: rename d1/d12/d1a/l42 to d1/d10/d26/l47 0 2024-03-11T19:55:53.151 INFO:tasks.workunit.client.0.smithi107.stdout:3/248: stat d1/d19/f32 0 2024-03-11T19:55:53.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/235: symlink d4/d1c/d47/l5a 0 2024-03-11T19:55:53.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/236: chown d4/d1d/d1f 2 1 2024-03-11T19:55:53.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/237: dread - d4/d1d/d1f/d28/d2c/f49 zero size 2024-03-11T19:55:53.156 INFO:tasks.workunit.client.0.smithi107.stdout:9/238: rename d4/d22/d17/f19 to d4/d1d/d1f/d28/d2c/f5b 0 2024-03-11T19:55:53.157 INFO:tasks.workunit.client.0.smithi107.stdout:9/239: mkdir d4/d1d/d5c 0 2024-03-11T19:55:53.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/264: dwrite d6/d19/d1e/f29 [0,4194304] 0 2024-03-11T19:55:53.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/240: mkdir d4/d1d/d1f/d28/d5d 0 2024-03-11T19:55:53.186 INFO:tasks.workunit.client.0.smithi107.stdout:6/252: dwrite d10/d16/d2b/f43 [0,4194304] 0 2024-03-11T19:55:53.190 INFO:tasks.workunit.client.0.smithi107.stdout:3/249: dread d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:53.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/241: dwrite d1/fc [4194304,4194304] 0 2024-03-11T19:55:53.193 INFO:tasks.workunit.client.0.smithi107.stdout:1/265: dread d6/d19/d1e/d27/f38 [0,4194304] 0 2024-03-11T19:55:53.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/242: mknod d1/c4d 0 2024-03-11T19:55:53.196 INFO:tasks.workunit.client.0.smithi107.stdout:3/250: rmdir d1/d10/d21 39 2024-03-11T19:55:53.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/243: rename d1/l3c to d1/l4e 0 2024-03-11T19:55:53.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/244: mkdir d1/d1a/d34/d4f 0 2024-03-11T19:55:53.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/245: chown d1/d1a/d1e/d26/l3d 862 1 2024-03-11T19:55:53.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/293: dwrite d6/d21/f3a [0,4194304] 0 2024-03-11T19:55:53.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/294: fdatasync d6/d21/f3a 0 2024-03-11T19:55:53.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/295: write d6/ff [15098,86331] 0 2024-03-11T19:55:53.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/296: fdatasync d6/da/d16/f2c 0 2024-03-11T19:55:53.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/241: dwrite d4/d1d/d1f/d28/f42 [0,4194304] 0 2024-03-11T19:55:53.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/242: dread - d4/d1d/d1f/d28/d2c/f49 zero size 2024-03-11T19:55:53.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/285: sync 2024-03-11T19:55:53.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/225: sync 2024-03-11T19:55:53.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/246: sync 2024-03-11T19:55:53.213 INFO:tasks.workunit.client.0.smithi107.stdout:7/297: truncate d6/ff 283147 0 2024-03-11T19:55:53.216 INFO:tasks.workunit.client.0.smithi107.stdout:9/243: getdents d4/d1d/d4b 0 2024-03-11T19:55:53.216 INFO:tasks.workunit.client.0.smithi107.stdout:9/244: chown d4/f16 428871 1 2024-03-11T19:55:53.216 INFO:tasks.workunit.client.0.smithi107.stdout:9/245: stat d4/f27 0 2024-03-11T19:55:53.217 INFO:tasks.workunit.client.0.smithi107.stdout:9/246: readlink d4/lf 0 2024-03-11T19:55:53.222 INFO:tasks.workunit.client.0.smithi107.stdout:0/247: getdents de/d11/d23/d26 0 2024-03-11T19:55:53.222 INFO:tasks.workunit.client.0.smithi107.stdout:7/298: symlink d6/da/d16/l5b 0 2024-03-11T19:55:53.222 INFO:tasks.workunit.client.0.smithi107.stdout:0/248: write de/d11/d23/d26/f42 [722635,93884] 0 2024-03-11T19:55:53.222 INFO:tasks.workunit.client.0.smithi107.stdout:1/266: dread d6/d19/f43 [0,4194304] 0 2024-03-11T19:55:53.226 INFO:tasks.workunit.client.0.smithi107.stdout:1/267: write d6/f48 [716786,117970] 0 2024-03-11T19:55:53.231 INFO:tasks.workunit.client.0.smithi107.stdout:9/247: rename d4/d1d/d1f/d28/d2c/l45 to d4/d1d/d1f/d28/l5e 0 2024-03-11T19:55:53.231 INFO:tasks.workunit.client.0.smithi107.stdout:6/253: read f0 [87092,55220] 0 2024-03-11T19:55:53.231 INFO:tasks.workunit.client.0.smithi107.stdout:6/254: chown d10/d19/d38/l4b 1 1 2024-03-11T19:55:53.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/268: rename d6/l44 to d6/d19/d1e/l52 0 2024-03-11T19:55:53.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/269: chown d6/f10 212993 1 2024-03-11T19:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/251: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout:4/226: dread d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/248: rename d4/c5 to d4/d1d/d4b/c5f 0 2024-03-11T19:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/252: truncate d1/d19/f32 18968 0 2024-03-11T19:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/249: getdents de/d11/d23 0 2024-03-11T19:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/250: readlink l7 0 2024-03-11T19:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout:6/255: symlink d10/d16/d1f/l57 0 2024-03-11T19:55:53.273 INFO:tasks.workunit.client.0.smithi107.stdout:2/246: dwrite d1/f40 [0,4194304] 0 2024-03-11T19:55:53.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/247: read d1/d1a/f2c [487295,104037] 0 2024-03-11T19:55:53.281 INFO:tasks.workunit.client.0.smithi107.stdout:0/251: dread fd [0,4194304] 0 2024-03-11T19:55:53.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/253: read d1/d10/d26/f30 [212107,97580] 0 2024-03-11T19:55:53.285 INFO:tasks.workunit.client.0.smithi107.stdout:3/254: write d1/d10/d13/f3b [1170889,63625] 0 2024-03-11T19:55:53.285 INFO:tasks.workunit.client.0.smithi107.stdout:3/255: stat d1/d10/d2a/f38 0 2024-03-11T19:55:53.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/252: symlink de/d11/d23/d24/l4e 0 2024-03-11T19:55:53.286 INFO:tasks.workunit.client.0.smithi107.stdout:5/286: dwrite da/d14/f3a [0,4194304] 0 2024-03-11T19:55:53.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/253: rmdir de/d11/d23 39 2024-03-11T19:55:53.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/256: dread d10/d16/f1e [0,4194304] 0 2024-03-11T19:55:53.291 INFO:tasks.workunit.client.0.smithi107.stdout:5/287: mkdir da/d31/d58 0 2024-03-11T19:55:53.291 INFO:tasks.workunit.client.0.smithi107.stdout:4/227: dwrite d4/d7/d1f/d2b/d24/d26/f43 [0,4194304] 0 2024-03-11T19:55:53.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/270: dwrite d6/f1b [0,4194304] 0 2024-03-11T19:55:53.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/228: symlink d4/d7/d1f/d2b/d24/d26/d29/l48 0 2024-03-11T19:55:53.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/229: stat d4/d7/db/lf 0 2024-03-11T19:55:53.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/271: symlink d6/d19/d1e/d27/l53 0 2024-03-11T19:55:53.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/272: write d6/d19/d1e/f4d [113820,32229] 0 2024-03-11T19:55:53.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/249: dwrite d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:53.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/230: dread d4/d7/db/f21 [0,4194304] 0 2024-03-11T19:55:53.326 INFO:tasks.workunit.client.0.smithi107.stdout:2/248: dwrite d1/f15 [0,4194304] 0 2024-03-11T19:55:53.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/231: truncate d4/d7/f22 341819 0 2024-03-11T19:55:53.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/232: readlink d4/d7/d1f/d2b/d24/d26/d29/l46 0 2024-03-11T19:55:53.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/233: creat d4/d7/db/f49 x:0 0 0 2024-03-11T19:55:53.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/250: dread d4/d1d/d1f/d28/d2c/f34 [0,4194304] 0 2024-03-11T19:55:53.338 INFO:tasks.workunit.client.0.smithi107.stdout:5/288: dwrite da/d31/d42/f23 [0,4194304] 0 2024-03-11T19:55:53.348 INFO:tasks.workunit.client.0.smithi107.stdout:5/289: rename da/d14/d19 to da/d14/d38/d59 0 2024-03-11T19:55:53.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/290: link da/d31/d42/f28 da/d14/d38/d59/d54/f5a 0 2024-03-11T19:55:53.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/291: read - da/d31/d42/f40 zero size 2024-03-11T19:55:53.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/292: write da/d14/d18/f51 [474697,17977] 0 2024-03-11T19:55:53.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/293: chown da 113435979 1 2024-03-11T19:55:53.351 INFO:tasks.workunit.client.0.smithi107.stdout:0/254: dread de/f2f [0,4194304] 0 2024-03-11T19:55:53.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/256: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:53.355 INFO:tasks.workunit.client.0.smithi107.stdout:0/255: truncate de/f13 802107 0 2024-03-11T19:55:53.355 INFO:tasks.workunit.client.0.smithi107.stdout:5/294: link da/d31/d42/l4e da/d14/d38/d59/l5b 0 2024-03-11T19:55:53.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/257: dwrite d10/d16/f37 [0,4194304] 0 2024-03-11T19:55:53.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/257: unlink d1/d12/d1a/f37 0 2024-03-11T19:55:53.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/258: mkdir d1/d10/d21/d43/d48 0 2024-03-11T19:55:53.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/234: dwrite d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:53.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/235: dread - d4/d7/d2f/d3c/f47 zero size 2024-03-11T19:55:53.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/236: truncate d4/d7/db/f21 108024 0 2024-03-11T19:55:53.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/237: stat d4/d7/db/c12 0 2024-03-11T19:55:53.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/251: dwrite f1 [0,4194304] 0 2024-03-11T19:55:53.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/238: symlink d4/d7/d2f/d33/d3a/l4a 0 2024-03-11T19:55:53.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/252: creat d4/d1c/d47/f60 x:0 0 0 2024-03-11T19:55:53.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/253: dread - d4/f58 zero size 2024-03-11T19:55:53.409 INFO:tasks.workunit.client.0.smithi107.stdout:5/295: dwrite da/d14/f3a [0,4194304] 0 2024-03-11T19:55:53.409 INFO:tasks.workunit.client.0.smithi107.stdout:5/296: fsync da/f36 0 2024-03-11T19:55:53.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/239: dread f3 [0,4194304] 0 2024-03-11T19:55:53.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/240: fsync d4/d7/f17 0 2024-03-11T19:55:53.418 INFO:tasks.workunit.client.0.smithi107.stdout:4/241: getdents d4/d7/d1f/d2b/d24/d26/d29/d36 0 2024-03-11T19:55:53.421 INFO:tasks.workunit.client.0.smithi107.stdout:6/258: dread d10/d19/f23 [0,4194304] 0 2024-03-11T19:55:53.421 INFO:tasks.workunit.client.0.smithi107.stdout:6/259: chown d10/d19/d38/d40 1037515 1 2024-03-11T19:55:53.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/256: dwrite fd [4194304,4194304] 0 2024-03-11T19:55:53.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/260: getdents d10/d3d 0 2024-03-11T19:55:53.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/257: mkdir de/d11/d23/d24/d4f 0 2024-03-11T19:55:53.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/258: getdents de/d11/d23 0 2024-03-11T19:55:53.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/259: getdents de/d11/d23/d24/d4f 0 2024-03-11T19:55:53.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/261: readlink d10/d16/d2b/l45 0 2024-03-11T19:55:53.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/260: creat de/d11/d23/d32/d34/d39/d3a/f50 x:0 0 0 2024-03-11T19:55:53.431 INFO:tasks.workunit.client.0.smithi107.stdout:2/249: dwrite d1/f2b [0,4194304] 0 2024-03-11T19:55:53.432 INFO:tasks.workunit.client.0.smithi107.stdout:3/259: dread d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:53.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/297: write da/f1a [3112869,32444] 0 2024-03-11T19:55:53.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/260: mkdir d1/d12/d49 0 2024-03-11T19:55:53.439 INFO:tasks.workunit.client.0.smithi107.stdout:5/298: rmdir da/d31/d58 0 2024-03-11T19:55:53.441 INFO:tasks.workunit.client.0.smithi107.stdout:0/261: link de/d11/f4c de/d11/f51 0 2024-03-11T19:55:53.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/262: rename de/d11/l38 to de/d11/d23/d32/d34/d39/l52 0 2024-03-11T19:55:53.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/245: sync 2024-03-11T19:55:53.448 INFO:tasks.workunit.client.0.smithi107.stdout:7/299: sync 2024-03-11T19:55:53.451 INFO:tasks.workunit.client.0.smithi107.stdout:9/254: dwrite d4/d22/f2d [0,4194304] 0 2024-03-11T19:55:53.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/262: dread d10/d16/f20 [0,4194304] 0 2024-03-11T19:55:53.455 INFO:tasks.workunit.client.0.smithi107.stdout:2/250: write d1/f2b [2322246,68661] 0 2024-03-11T19:55:53.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/300: rename d6/da/d47 to d6/da/d16/d5c 0 2024-03-11T19:55:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/301: write d6/d9/d13/d1b/f1c [3253818,54934] 0 2024-03-11T19:55:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/302: readlink d6/da/d16/l5b 0 2024-03-11T19:55:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/251: mknod d1/d1a/c50 0 2024-03-11T19:55:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/255: rename d4/lf to d4/d1c/l61 0 2024-03-11T19:55:53.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/252: creat d1/d1a/d1e/d41/f51 x:0 0 0 2024-03-11T19:55:53.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/253: symlink d1/d1a/d1e/d26/l52 0 2024-03-11T19:55:53.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/254: write d1/d1a/f2c [1457761,19110] 0 2024-03-11T19:55:53.461 INFO:tasks.workunit.client.0.smithi107.stdout:2/255: symlink d1/d1a/d34/d4f/l53 0 2024-03-11T19:55:53.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/273: sync 2024-03-11T19:55:53.484 INFO:tasks.workunit.client.0.smithi107.stdout:1/274: fdatasync d6/d19/d1e/d27/d2c/f30 0 2024-03-11T19:55:53.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/246: dread dd/d13/d18/f1c [4194304,4194304] 0 2024-03-11T19:55:53.484 INFO:tasks.workunit.client.0.smithi107.stdout:0/263: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:53.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/264: creat de/d11/d23/d32/d34/d39/f53 x:0 0 0 2024-03-11T19:55:53.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/247: rename dd/d13/d18/f40 to dd/d13/d22/f44 0 2024-03-11T19:55:53.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/248: chown dd/d16/f1e 173 1 2024-03-11T19:55:53.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/256: dread d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:53.495 INFO:tasks.workunit.client.0.smithi107.stdout:9/257: rename d4/d1c/d47/f60 to d4/d1d/d1f/d28/d2c/f62 0 2024-03-11T19:55:53.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/258: symlink d4/d1d/d5c/l63 0 2024-03-11T19:55:53.507 INFO:tasks.workunit.client.0.smithi107.stdout:5/299: dwrite da/d14/d18/f1c [0,4194304] 0 2024-03-11T19:55:53.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/263: read d10/d16/f1c [329585,70324] 0 2024-03-11T19:55:53.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/264: chown d10/d16/d24/d2e/f36 2171550 1 2024-03-11T19:55:53.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/303: dwrite d6/d9/d13/d1b/f3c [0,4194304] 0 2024-03-11T19:55:53.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/304: write d6/da/d16/d5c/f54 [1326405,10239] 0 2024-03-11T19:55:53.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/261: dwrite d1/d10/d13/f3b [0,4194304] 0 2024-03-11T19:55:53.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/262: getdents d1/d45 0 2024-03-11T19:55:53.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/265: dread de/f3f [0,4194304] 0 2024-03-11T19:55:53.520 INFO:tasks.workunit.client.0.smithi107.stdout:7/305: symlink d6/d9/d13/l5d 0 2024-03-11T19:55:53.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/300: dread da/d14/d38/d59/f1d [0,4194304] 0 2024-03-11T19:55:53.522 INFO:tasks.workunit.client.0.smithi107.stdout:7/306: creat d6/d9/d13/f5e x:0 0 0 2024-03-11T19:55:53.523 INFO:tasks.workunit.client.0.smithi107.stdout:2/256: dwrite d1/f2b [0,4194304] 0 2024-03-11T19:55:53.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/266: creat de/d11/d23/d32/f54 x:0 0 0 2024-03-11T19:55:53.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/263: mkdir d1/d10/d13/d4a 0 2024-03-11T19:55:53.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/257: rmdir d1/d1a/d1e/d26 39 2024-03-11T19:55:53.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/258: truncate d1/d1a/d1e/d26/f32 9357192 0 2024-03-11T19:55:53.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/259: chown d1/d1a/d1e/d26/c2a 508024 1 2024-03-11T19:55:53.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/260: chown d1/d1a/d1e/d41/l44 3 1 2024-03-11T19:55:53.530 INFO:tasks.workunit.client.0.smithi107.stdout:3/264: symlink d1/d10/d26/l4b 0 2024-03-11T19:55:53.543 INFO:tasks.workunit.client.0.smithi107.stdout:1/275: dwrite d6/d28/f3f [0,4194304] 0 2024-03-11T19:55:53.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/265: dread d10/d16/f1e [0,4194304] 0 2024-03-11T19:55:53.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/249: dwrite dd/d13/f3d [0,4194304] 0 2024-03-11T19:55:53.556 INFO:tasks.workunit.client.0.smithi107.stdout:2/261: dread d1/d1d/f3e [0,4194304] 0 2024-03-11T19:55:53.557 INFO:tasks.workunit.client.0.smithi107.stdout:2/262: fdatasync d1/fc 0 2024-03-11T19:55:53.558 INFO:tasks.workunit.client.0.smithi107.stdout:2/263: link d1/d1a/f2c d1/d21/d35/f54 0 2024-03-11T19:55:53.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/264: mkdir d1/d1a/d1e/d55 0 2024-03-11T19:55:53.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/265: dread d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:53.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/266: chown d1/d19/d2d/l3a 12 1 2024-03-11T19:55:53.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/307: dread d6/da/d16/d5c/f54 [0,4194304] 0 2024-03-11T19:55:53.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/301: dwrite da/d14/d38/d59/f1d [0,4194304] 0 2024-03-11T19:55:53.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/308: truncate d6/d9/d13/d1b/f1c 4726080 0 2024-03-11T19:55:53.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/309: read d6/d9/d13/d1b/f1c [3134235,58223] 0 2024-03-11T19:55:53.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/302: symlink da/d14/l5c 0 2024-03-11T19:55:53.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/303: dread - da/d14/d38/d59/d54/f57 zero size 2024-03-11T19:55:53.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/266: dwrite d10/d16/d24/d2e/f36 [0,4194304] 0 2024-03-11T19:55:53.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/276: dwrite d6/f15 [0,4194304] 0 2024-03-11T19:55:53.603 INFO:tasks.workunit.client.0.smithi107.stdout:1/277: creat d6/f54 x:0 0 0 2024-03-11T19:55:53.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/278: fdatasync d6/f10 0 2024-03-11T19:55:53.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/279: chown d6/f1b 7259777 1 2024-03-11T19:55:53.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/267: creat d10/d16/d24/f58 x:0 0 0 2024-03-11T19:55:53.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/268: creat d10/d16/d2b/f59 x:0 0 0 2024-03-11T19:55:53.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/269: write d10/d16/d24/f27 [994297,87711] 0 2024-03-11T19:55:53.606 INFO:tasks.workunit.client.0.smithi107.stdout:1/280: creat d6/f55 x:0 0 0 2024-03-11T19:55:53.606 INFO:tasks.workunit.client.0.smithi107.stdout:1/281: chown d6/f10 8365083 1 2024-03-11T19:55:53.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/265: dread d1/f2b [0,4194304] 0 2024-03-11T19:55:53.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/266: dread - d1/d1a/d1e/d26/f47 zero size 2024-03-11T19:55:53.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/267: rename d1/c19 to d1/d1a/d1e/d26/c56 0 2024-03-11T19:55:53.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/268: dread - d1/d1a/d1e/d41/f51 zero size 2024-03-11T19:55:53.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/270: fdatasync d10/d16/d24/f27 0 2024-03-11T19:55:53.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/271: fdatasync f8 0 2024-03-11T19:55:53.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/272: creat d10/d3d/f5a x:0 0 0 2024-03-11T19:55:53.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/273: dread - d10/d3d/f5a zero size 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/250: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/251: stat dd/d13/f34 0 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/252: fdatasync dd/d13/f3d 0 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/253: chown dd/d13/d18/c23 2043955 1 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/267: dwrite d1/d19/f32 [0,4194304] 0 2024-03-11T19:55:53.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/268: chown d1/d10/c11 192913090 1 2024-03-11T19:55:53.619 INFO:tasks.workunit.client.0.smithi107.stdout:8/254: chown dd/l12 26 1 2024-03-11T19:55:53.619 INFO:tasks.workunit.client.0.smithi107.stdout:8/255: dread - dd/d13/d22/f42 zero size 2024-03-11T19:55:53.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/256: creat dd/d13/d32/f45 x:0 0 0 2024-03-11T19:55:53.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/257: readlink dd/de/d1b/d2c/l30 0 2024-03-11T19:55:53.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/258: truncate dd/de/d1b/d2c/f2d 1996821 0 2024-03-11T19:55:53.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/259: readlink l7 0 2024-03-11T19:55:53.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/260: chown dd/de/c11 1186 1 2024-03-11T19:55:53.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/261: mkdir dd/d13/d22/d46 0 2024-03-11T19:55:53.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/262: write dd/d13/f34 [355948,33264] 0 2024-03-11T19:55:53.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/263: mkdir dd/de/d24/d28/d47 0 2024-03-11T19:55:53.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/264: creat dd/de/f48 x:0 0 0 2024-03-11T19:55:53.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/282: dread d6/f1b [0,4194304] 0 2024-03-11T19:55:53.633 INFO:tasks.workunit.client.0.smithi107.stdout:2/269: write d1/f40 [649898,92764] 0 2024-03-11T19:55:53.633 INFO:tasks.workunit.client.0.smithi107.stdout:2/270: stat d1/d1a/d1e/d26/l52 0 2024-03-11T19:55:53.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/265: rename dd/d13/d22 to dd/de/d49 0 2024-03-11T19:55:53.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/271: truncate d1/fc 1254893 0 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/266: creat dd/de/d24/d28/f4a x:0 0 0 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/310: dwrite d6/f2f [0,4194304] 0 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/311: chown d6/d21/l3e 9578 1 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/312: chown d6/d9/d4e 182745580 1 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/313: creat d6/f5f x:0 0 0 2024-03-11T19:55:53.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/314: read - d6/d9/d4e/f59 zero size 2024-03-11T19:55:53.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/304: dwrite da/f1a [0,4194304] 0 2024-03-11T19:55:53.660 INFO:tasks.workunit.client.0.smithi107.stdout:1/283: dread d6/d19/d1e/f29 [0,4194304] 0 2024-03-11T19:55:53.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/315: dread d6/da/d16/f3f [0,4194304] 0 2024-03-11T19:55:53.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/316: write d6/d9/d13/f51 [277391,32911] 0 2024-03-11T19:55:53.677 INFO:tasks.workunit.client.0.smithi107.stdout:2/272: dread d1/f2b [0,4194304] 0 2024-03-11T19:55:53.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/267: write f4 [6196988,100511] 0 2024-03-11T19:55:53.681 INFO:tasks.workunit.client.0.smithi107.stdout:3/269: dwrite d1/d12/d1a/f20 [0,4194304] 0 2024-03-11T19:55:53.681 INFO:tasks.workunit.client.0.smithi107.stdout:3/270: chown d1/d10/d21/d43/d48 4502372 1 2024-03-11T19:55:53.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/273: symlink d1/d1a/d1e/d41/l57 0 2024-03-11T19:55:53.686 INFO:tasks.workunit.client.0.smithi107.stdout:8/268: mkdir dd/de/d1b/d4b 0 2024-03-11T19:55:53.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/274: symlink d1/d1a/d1e/d26/l58 0 2024-03-11T19:55:53.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/305: dwrite f4 [4194304,4194304] 0 2024-03-11T19:55:53.690 INFO:tasks.workunit.client.0.smithi107.stdout:3/271: rename d1/d12 to d1/d10/d13/d4a/d4c 0 2024-03-11T19:55:53.691 INFO:tasks.workunit.client.0.smithi107.stdout:5/306: unlink da/d14/d18/d44/f49 0 2024-03-11T19:55:53.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/307: symlink da/d14/d18/d1e/l5d 0 2024-03-11T19:55:53.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/269: dwrite dd/de/d1b/d20/f26 [0,4194304] 0 2024-03-11T19:55:53.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/242: sync 2024-03-11T19:55:53.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/270: creat dd/f4c x:0 0 0 2024-03-11T19:55:53.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/317: dwrite d6/f2f [0,4194304] 0 2024-03-11T19:55:53.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/318: write d6/ff [203002,126880] 0 2024-03-11T19:55:53.747 INFO:tasks.workunit.client.0.smithi107.stdout:3/272: dwrite d1/d10/d26/f30 [0,4194304] 0 2024-03-11T19:55:53.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/271: write dd/d16/f33 [7043875,35574] 0 2024-03-11T19:55:53.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/308: dwrite da/d14/f3a [4194304,4194304] 0 2024-03-11T19:55:53.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/309: chown da/d31/d42/l4e 1864 1 2024-03-11T19:55:53.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/273: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:53.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/274: truncate d1/d10/d13/f3b 2361450 0 2024-03-11T19:55:53.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/319: dwrite d6/da/d16/f2e [0,4194304] 0 2024-03-11T19:55:53.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/320: chown d6/f17 9 1 2024-03-11T19:55:53.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/321: fdatasync d6/d9/f30 0 2024-03-11T19:55:53.783 INFO:tasks.workunit.client.0.smithi107.stdout:4/243: dwrite d4/d7/d2f/d3c/f47 [0,4194304] 0 2024-03-11T19:55:53.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/244: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/c4b 0 2024-03-11T19:55:53.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/322: dread d6/da/d16/f2c [0,4194304] 0 2024-03-11T19:55:53.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/245: dread d4/d7/d1f/d2b/d24/d26/d29/f3f [0,4194304] 0 2024-03-11T19:55:53.796 INFO:tasks.workunit.client.0.smithi107.stdout:5/310: dwrite da/fb [0,4194304] 0 2024-03-11T19:55:53.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/275: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:53.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/246: dread d4/d7/f22 [0,4194304] 0 2024-03-11T19:55:53.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/247: fdatasync d4/d7/db/f49 0 2024-03-11T19:55:53.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/248: dread - d4/d7/db/f49 zero size 2024-03-11T19:55:53.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/249: getdents d4/d7/d1f/d2b/d24/d26/d29 0 2024-03-11T19:55:53.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/311: dread da/d14/d38/f3f [0,4194304] 0 2024-03-11T19:55:53.830 INFO:tasks.workunit.client.0.smithi107.stdout:3/276: dwrite d1/fa [0,4194304] 0 2024-03-11T19:55:53.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/259: sync 2024-03-11T19:55:53.831 INFO:tasks.workunit.client.0.smithi107.stdout:0/267: sync 2024-03-11T19:55:53.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/260: chown d4/d9/d4d 75 1 2024-03-11T19:55:53.832 INFO:tasks.workunit.client.0.smithi107.stdout:3/277: unlink d1/d10/d21/f28 0 2024-03-11T19:55:53.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/261: mkdir d4/d1d/d5c/d64 0 2024-03-11T19:55:53.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/268: mkdir de/d11/d23/d32/d34/d39/d55 0 2024-03-11T19:55:53.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/269: dread - de/d11/f2c zero size 2024-03-11T19:55:53.834 INFO:tasks.workunit.client.0.smithi107.stdout:9/262: mknod d4/d1c/d47/c65 0 2024-03-11T19:55:53.835 INFO:tasks.workunit.client.0.smithi107.stdout:9/263: stat d4/d1d/d1f/d28/c57 0 2024-03-11T19:55:53.835 INFO:tasks.workunit.client.0.smithi107.stdout:0/270: rmdir de/d11/d23 39 2024-03-11T19:55:53.835 INFO:tasks.workunit.client.0.smithi107.stdout:9/264: symlink d4/d1d/d1f/d28/l66 0 2024-03-11T19:55:53.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/271: write de/d11/d23/d32/f54 [1016573,107918] 0 2024-03-11T19:55:53.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/265: creat d4/d1d/d1f/d28/f67 x:0 0 0 2024-03-11T19:55:53.838 INFO:tasks.workunit.client.0.smithi107.stdout:9/266: truncate d4/f58 466477 0 2024-03-11T19:55:53.838 INFO:tasks.workunit.client.0.smithi107.stdout:9/267: chown d4/d1c/d37/f50 406610955 1 2024-03-11T19:55:53.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/312: dwrite da/d14/d18/f51 [0,4194304] 0 2024-03-11T19:55:53.839 INFO:tasks.workunit.client.0.smithi107.stdout:0/272: mknod de/d11/d23/c56 0 2024-03-11T19:55:53.841 INFO:tasks.workunit.client.0.smithi107.stdout:9/268: rmdir d4/d1d/d4b 39 2024-03-11T19:55:53.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/313: rename da/f1a to da/d14/f5e 0 2024-03-11T19:55:53.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/314: write da/d14/d18/f27 [433312,45573] 0 2024-03-11T19:55:53.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/315: rmdir da/d14/d38/d46 39 2024-03-11T19:55:53.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/316: dread - da/d31/d42/f40 zero size 2024-03-11T19:55:53.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/317: chown da/d14/d18/f51 78776026 1 2024-03-11T19:55:53.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/318: dread - da/d14/d38/d59/d54/f57 zero size 2024-03-11T19:55:53.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/319: dread - da/d31/f4f zero size 2024-03-11T19:55:53.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/320: mkdir da/d14/d38/d59/d5f 0 2024-03-11T19:55:53.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/321: mknod da/d31/d42/c60 0 2024-03-11T19:55:53.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/322: write da/d14/d18/f27 [202897,125611] 0 2024-03-11T19:55:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/323: rename da/d14/f35 to da/d14/d38/d46/f61 0 2024-03-11T19:55:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/278: write d1/d10/d13/d4a/d4c/d1a/f20 [3214219,44031] 0 2024-03-11T19:55:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/279: fdatasync d1/d19/f44 0 2024-03-11T19:55:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/324: dread - da/d31/d42/f40 zero size 2024-03-11T19:55:53.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/280: fsync d1/d10/d2a/f38 0 2024-03-11T19:55:53.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/325: write da/d14/d38/f3f [78571,105677] 0 2024-03-11T19:55:53.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/269: dread d4/f16 [0,4194304] 0 2024-03-11T19:55:53.860 INFO:tasks.workunit.client.0.smithi107.stdout:0/273: read de/d11/d23/d24/f2d [1277742,129693] 0 2024-03-11T19:55:53.860 INFO:tasks.workunit.client.0.smithi107.stdout:5/326: symlink da/d14/d38/l62 0 2024-03-11T19:55:53.860 INFO:tasks.workunit.client.0.smithi107.stdout:5/327: chown da/d14/d38 69588 1 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/274: chown de/d11/f2c 3421 1 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:5/328: chown da/d14/d18/f51 17863 1 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/275: truncate de/f40 627490 0 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/323: dwrite d6/d9/d13/f51 [0,4194304] 0 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/324: readlink d6/d9/l34 0 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/325: stat d6/da/d49 0 2024-03-11T19:55:53.861 INFO:tasks.workunit.client.0.smithi107.stdout:9/270: write d4/d1d/d1f/d28/d2c/f46 [494487,100785] 0 2024-03-11T19:55:53.863 INFO:tasks.workunit.client.0.smithi107.stdout:5/329: unlink da/d14/f45 0 2024-03-11T19:55:53.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/326: mkdir d6/d9/d13/d60 0 2024-03-11T19:55:53.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/276: read de/f13 [484484,83024] 0 2024-03-11T19:55:53.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/271: creat d4/d22/f68 x:0 0 0 2024-03-11T19:55:53.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/272: chown d4/d22/d17/f40 12762 1 2024-03-11T19:55:53.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/330: symlink da/l63 0 2024-03-11T19:55:53.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/327: rmdir d6/d9/d57 39 2024-03-11T19:55:53.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/328: chown d6/c46 257915107 1 2024-03-11T19:55:53.869 INFO:tasks.workunit.client.0.smithi107.stdout:9/273: symlink d4/d1d/d1f/l69 0 2024-03-11T19:55:53.881 INFO:tasks.workunit.client.0.smithi107.stdout:9/274: unlink d4/d22/d17/l24 0 2024-03-11T19:55:53.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/331: rename da/d14/d38/f56 to da/d14/d38/d59/f64 0 2024-03-11T19:55:53.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/275: truncate d4/d22/d17/f40 6303167 0 2024-03-11T19:55:53.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/276: write d4/d1c/d37/f50 [1462979,74800] 0 2024-03-11T19:55:53.882 INFO:tasks.workunit.client.0.smithi107.stdout:5/332: unlink da/d14/d38/d59/d54/f5a 0 2024-03-11T19:55:53.883 INFO:tasks.workunit.client.0.smithi107.stdout:9/277: rename l3 to d4/d22/l6a 0 2024-03-11T19:55:53.883 INFO:tasks.workunit.client.0.smithi107.stdout:9/278: stat d4/d1c/d47/c48 0 2024-03-11T19:55:53.885 INFO:tasks.workunit.client.0.smithi107.stdout:9/279: rename d4/d1c/l4a to d4/d1d/d1f/d28/d5d/l6b 0 2024-03-11T19:55:53.891 INFO:tasks.workunit.client.0.smithi107.stdout:4/250: dwrite d4/d7/d1f/d2b/d24/d26/d29/f3f [0,4194304] 0 2024-03-11T19:55:53.893 INFO:tasks.workunit.client.0.smithi107.stdout:4/251: truncate d4/d7/f2c 3846070 0 2024-03-11T19:55:53.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/252: read d4/d7/d2f/d33/d3a/f44 [416032,112272] 0 2024-03-11T19:55:53.902 INFO:tasks.workunit.client.0.smithi107.stdout:4/253: fdatasync d4/d7/d1f/d2b/d19/f23 0 2024-03-11T19:55:53.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/254: symlink d4/d7/d1f/d2b/d19/l4c 0 2024-03-11T19:55:53.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/255: fsync f3 0 2024-03-11T19:55:53.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/277: dwrite f3 [4194304,4194304] 0 2024-03-11T19:55:53.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/329: dread d6/da/f1f [0,4194304] 0 2024-03-11T19:55:53.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/278: link de/d11/d23/d24/f2d de/d11/d23/d32/d34/d39/d3a/f57 0 2024-03-11T19:55:53.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/279: write de/f3f [737462,12483] 0 2024-03-11T19:55:53.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/280: readlink l1 0 2024-03-11T19:55:53.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/281: fdatasync de/f13 0 2024-03-11T19:55:53.923 INFO:tasks.workunit.client.0.smithi107.stdout:7/330: mkdir d6/d9/d13/d60/d61 0 2024-03-11T19:55:53.925 INFO:tasks.workunit.client.0.smithi107.stdout:7/331: link d6/da/d16/c3d d6/d9/d13/d60/d61/c62 0 2024-03-11T19:55:53.925 INFO:tasks.workunit.client.0.smithi107.stdout:9/280: dread d4/d1d/d1f/d28/d2c/f5b [0,4194304] 0 2024-03-11T19:55:53.925 INFO:tasks.workunit.client.0.smithi107.stdout:3/281: dwrite d1/d10/d13/d4a/d4c/f2f [4194304,4194304] 0 2024-03-11T19:55:53.925 INFO:tasks.workunit.client.0.smithi107.stdout:3/282: chown d1/d19/c3f 35987412 1 2024-03-11T19:55:53.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/333: dwrite da/d31/f4f [0,4194304] 0 2024-03-11T19:55:53.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/332: symlink d6/da/d16/d5c/l63 0 2024-03-11T19:55:53.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/281: unlink d4/d1c/d47/l5a 0 2024-03-11T19:55:53.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/283: mkdir d1/d19/d2d/d4d 0 2024-03-11T19:55:53.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/284: chown d1/d10/d13/d4a/d4c/f2f 65548099 1 2024-03-11T19:55:53.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/285: fsync d1/d10/d2a/f38 0 2024-03-11T19:55:53.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/334: creat da/d14/d18/d1e/f65 x:0 0 0 2024-03-11T19:55:53.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/274: sync 2024-03-11T19:55:53.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/286: mkdir d1/d10/d4e 0 2024-03-11T19:55:53.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/275: readlink d10/d16/d2b/l47 0 2024-03-11T19:55:53.943 INFO:tasks.workunit.client.0.smithi107.stdout:1/284: sync 2024-03-11T19:55:53.943 INFO:tasks.workunit.client.0.smithi107.stdout:1/285: write d6/d19/d1e/f21 [4872439,47889] 0 2024-03-11T19:55:53.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/275: sync 2024-03-11T19:55:53.948 INFO:tasks.workunit.client.0.smithi107.stdout:8/272: sync 2024-03-11T19:55:53.948 INFO:tasks.workunit.client.0.smithi107.stdout:8/273: write dd/de/d49/f42 [48902,5217] 0 2024-03-11T19:55:53.948 INFO:tasks.workunit.client.0.smithi107.stdout:8/274: chown dd/de/d1b/d2c/l3e 183054 1 2024-03-11T19:55:53.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/275: write dd/de/d49/f42 [54985,28789] 0 2024-03-11T19:55:53.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/286: unlink d6/d19/l36 0 2024-03-11T19:55:53.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/287: mknod d6/d19/d1e/d27/c56 0 2024-03-11T19:55:53.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/276: symlink dd/de/d24/d28/d47/l4d 0 2024-03-11T19:55:53.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/277: creat dd/de/f4e x:0 0 0 2024-03-11T19:55:53.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/278: creat dd/de/d49/d46/f4f x:0 0 0 2024-03-11T19:55:53.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/279: rename dd/d13/d18/f1c to dd/d13/d18/f50 0 2024-03-11T19:55:53.983 INFO:tasks.workunit.client.0.smithi107.stdout:5/335: write da/d31/d42/f13 [1042062,30588] 0 2024-03-11T19:55:53.986 INFO:tasks.workunit.client.0.smithi107.stdout:5/336: mknod da/d31/c66 0 2024-03-11T19:55:53.986 INFO:tasks.workunit.client.0.smithi107.stdout:5/337: readlink da/d14/l5c 0 2024-03-11T19:55:53.986 INFO:tasks.workunit.client.0.smithi107.stdout:4/256: dwrite d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:55:53.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/338: mknod da/d14/d18/c67 0 2024-03-11T19:55:53.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/339: creat da/d14/f68 x:0 0 0 2024-03-11T19:55:53.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/340: chown da/d31/d42/f11 251775452 1 2024-03-11T19:55:53.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/341: fsync da/d14/d38/d59/d54/f57 0 2024-03-11T19:55:53.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/257: rmdir d4/d7/d1f 39 2024-03-11T19:55:54.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/258: getdents d4/d7/d2f/d3c 0 2024-03-11T19:55:54.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/259: readlink d4/d7/d1f/d2b/d24/d26/d29/d36/l37 0 2024-03-11T19:55:54.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/260: write d4/d7/d1f/d2b/d19/f2a [1190965,49351] 0 2024-03-11T19:55:54.011 INFO:tasks.workunit.client.0.smithi107.stdout:0/282: dwrite de/d11/d23/d32/f54 [0,4194304] 0 2024-03-11T19:55:54.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/333: dwrite d6/ff [0,4194304] 0 2024-03-11T19:55:54.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/282: dwrite d4/d1d/d1f/d28/d2c/f5b [4194304,4194304] 0 2024-03-11T19:55:54.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/283: stat d4/d9/fa 0 2024-03-11T19:55:54.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/284: fsync d4/d1d/d1f/d28/f67 0 2024-03-11T19:55:54.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/285: creat d4/d1d/d1f/d28/f6c x:0 0 0 2024-03-11T19:55:54.019 INFO:tasks.workunit.client.0.smithi107.stdout:7/334: symlink d6/d9/l64 0 2024-03-11T19:55:54.023 INFO:tasks.workunit.client.0.smithi107.stdout:3/287: dwrite d1/d10/d13/d4a/d4c/d1a/f20 [0,4194304] 0 2024-03-11T19:55:54.024 INFO:tasks.workunit.client.0.smithi107.stdout:3/288: dread - d1/d19/f44 zero size 2024-03-11T19:55:54.031 INFO:tasks.workunit.client.0.smithi107.stdout:7/335: mkdir d6/d9/d57/d65 0 2024-03-11T19:55:54.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/276: dwrite f8 [4194304,4194304] 0 2024-03-11T19:55:54.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/286: rename d4/d22 to d4/d1d/d1f/d28/d2c/d56/d6d 0 2024-03-11T19:55:54.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/288: dwrite d6/d19/d1e/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:54.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/289: link d6/d19/d1e/d27/d2c/l51 d6/d19/d3d/l57 0 2024-03-11T19:55:54.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/290: getdents d6/d28 0 2024-03-11T19:55:54.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/291: unlink d6/d19/d1e/d27/l53 0 2024-03-11T19:55:54.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/342: dread da/d14/d38/f3f [0,4194304] 0 2024-03-11T19:55:54.038 INFO:tasks.workunit.client.0.smithi107.stdout:5/343: creat da/d14/d38/d59/f69 x:0 0 0 2024-03-11T19:55:54.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/344: creat da/d14/d38/d59/d5f/f6a x:0 0 0 2024-03-11T19:55:54.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/280: dwrite dd/de/d1b/d2c/f2d [0,4194304] 0 2024-03-11T19:55:54.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/281: write dd/f2e [849763,84382] 0 2024-03-11T19:55:54.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/336: dread d6/da/f1f [0,4194304] 0 2024-03-11T19:55:54.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/337: read - d6/f5f zero size 2024-03-11T19:55:54.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/345: mknod da/d14/c6b 0 2024-03-11T19:55:54.053 INFO:tasks.workunit.client.0.smithi107.stdout:6/277: write d10/d16/f2d [2590230,127184] 0 2024-03-11T19:55:54.053 INFO:tasks.workunit.client.0.smithi107.stdout:6/278: creat d10/d16/d24/f5b x:0 0 0 2024-03-11T19:55:54.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/283: dwrite fd [4194304,4194304] 0 2024-03-11T19:55:54.058 INFO:tasks.workunit.client.0.smithi107.stdout:8/282: mkdir dd/d13/d32/d51 0 2024-03-11T19:55:54.058 INFO:tasks.workunit.client.0.smithi107.stdout:8/283: dread - dd/de/f48 zero size 2024-03-11T19:55:54.060 INFO:tasks.workunit.client.0.smithi107.stdout:5/346: creat da/d14/d43/f6c x:0 0 0 2024-03-11T19:55:54.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/292: dread d6/d28/f3f [0,4194304] 0 2024-03-11T19:55:54.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/293: chown d6/d28/l2f 424371 1 2024-03-11T19:55:54.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/347: rmdir da/d14/d18 39 2024-03-11T19:55:54.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/294: mkdir d6/d19/d1e/d27/d2c/d32/d58 0 2024-03-11T19:55:54.065 INFO:tasks.workunit.client.0.smithi107.stdout:1/295: fsync d6/d19/d1e/f29 0 2024-03-11T19:55:54.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/296: mknod d6/d19/c59 0 2024-03-11T19:55:54.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/348: rmdir da/d14/d43 39 2024-03-11T19:55:54.068 INFO:tasks.workunit.client.0.smithi107.stdout:4/261: dwrite d4/d7/f1d [0,4194304] 0 2024-03-11T19:55:54.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/262: mkdir d4/d7/d1f/d2b/d19/d4d 0 2024-03-11T19:55:54.070 INFO:tasks.workunit.client.0.smithi107.stdout:1/297: unlink d6/d19/f40 0 2024-03-11T19:55:54.071 INFO:tasks.workunit.client.0.smithi107.stdout:4/263: creat d4/d7/d2f/d3b/f4e x:0 0 0 2024-03-11T19:55:54.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/298: write d6/d19/d1e/f4d [584583,109764] 0 2024-03-11T19:55:54.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/299: symlink d6/d19/d1e/d27/d2c/d32/d58/l5a 0 2024-03-11T19:55:54.077 INFO:tasks.workunit.client.0.smithi107.stdout:1/300: mkdir d6/d5b 0 2024-03-11T19:55:54.078 INFO:tasks.workunit.client.0.smithi107.stdout:1/301: stat d6/d19/d1e/d27/f38 0 2024-03-11T19:55:54.079 INFO:tasks.workunit.client.0.smithi107.stdout:7/338: dread d6/d21/f3a [0,4194304] 0 2024-03-11T19:55:54.080 INFO:tasks.workunit.client.0.smithi107.stdout:7/339: dread - d6/d9/d4e/f59 zero size 2024-03-11T19:55:54.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/287: dwrite d4/d9/f15 [0,4194304] 0 2024-03-11T19:55:54.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/288: chown d4/d1c 26 1 2024-03-11T19:55:54.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/284: dread dd/de/f3b [0,4194304] 0 2024-03-11T19:55:54.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/285: write dd/d13/d18/f50 [2815862,7535] 0 2024-03-11T19:55:54.089 INFO:tasks.workunit.client.0.smithi107.stdout:9/289: unlink d4/d1d/d1f/d28/d2c/f3f 0 2024-03-11T19:55:54.089 INFO:tasks.workunit.client.0.smithi107.stdout:9/290: chown d4/d1d/d1f/d28/d2c/d56/d6d/l33 1300 1 2024-03-11T19:55:54.089 INFO:tasks.workunit.client.0.smithi107.stdout:9/291: creat d4/d1d/d1f/d28/f6e x:0 0 0 2024-03-11T19:55:54.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/340: getdents d6/d9/d4e 0 2024-03-11T19:55:54.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/292: mknod d4/d1d/d1f/c6f 0 2024-03-11T19:55:54.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/341: mknod d6/c66 0 2024-03-11T19:55:54.099 INFO:tasks.workunit.client.0.smithi107.stdout:5/349: write f4 [8089385,21571] 0 2024-03-11T19:55:54.099 INFO:tasks.workunit.client.0.smithi107.stdout:5/350: write da/d31/d42/f11 [3114676,124896] 0 2024-03-11T19:55:54.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/286: rmdir dd 39 2024-03-11T19:55:54.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/279: dwrite d10/d16/d24/d2e/f55 [0,4194304] 0 2024-03-11T19:55:54.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/351: mkdir da/d14/d18/d6d 0 2024-03-11T19:55:54.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/352: chown da/d14/d18/d1e/c34 3833 1 2024-03-11T19:55:54.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/342: dread d6/da/d16/f2b [0,4194304] 0 2024-03-11T19:55:54.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/343: truncate d6/da/d16/f2b 1039336 0 2024-03-11T19:55:54.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/287: creat dd/d13/f52 x:0 0 0 2024-03-11T19:55:54.115 INFO:tasks.workunit.client.0.smithi107.stdout:5/353: truncate da/d31/f4f 853967 0 2024-03-11T19:55:54.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/293: dread d4/f53 [0,4194304] 0 2024-03-11T19:55:54.121 INFO:tasks.workunit.client.0.smithi107.stdout:5/354: rmdir da/d14/d38 39 2024-03-11T19:55:54.121 INFO:tasks.workunit.client.0.smithi107.stdout:5/355: fsync da/f32 0 2024-03-11T19:55:54.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/356: write da/d14/d38/f3f [27476,83977] 0 2024-03-11T19:55:54.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/357: write da/d14/d38/d59/f69 [726110,56533] 0 2024-03-11T19:55:54.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/294: rmdir d4/d1d/d1f/d28/d2c/d56 39 2024-03-11T19:55:54.124 INFO:tasks.workunit.client.0.smithi107.stdout:4/264: dwrite d4/f6 [0,4194304] 0 2024-03-11T19:55:54.124 INFO:tasks.workunit.client.0.smithi107.stdout:4/265: dread - d4/d7/d2f/d3b/f4e zero size 2024-03-11T19:55:54.125 INFO:tasks.workunit.client.0.smithi107.stdout:5/358: unlink da/d14/c4a 0 2024-03-11T19:55:54.125 INFO:tasks.workunit.client.0.smithi107.stdout:5/359: stat da/d14/d38/d59/f64 0 2024-03-11T19:55:54.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/302: rename f4 to d6/d19/d1e/f5c 0 2024-03-11T19:55:54.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/288: rename dd to dd/de/d24/d28/d53 22 2024-03-11T19:55:54.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/289: chown dd/d16 48107355 1 2024-03-11T19:55:54.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/303: chown d6/d19/c22 13564281 1 2024-03-11T19:55:54.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/290: stat dd/de/d1b 0 2024-03-11T19:55:54.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/295: mknod d4/d9/d4d/c70 0 2024-03-11T19:55:54.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/266: mknod d4/d7/d1f/d2b/d24/d26/d29/c4f 0 2024-03-11T19:55:54.132 INFO:tasks.workunit.client.0.smithi107.stdout:5/360: mkdir da/d6e 0 2024-03-11T19:55:54.132 INFO:tasks.workunit.client.0.smithi107.stdout:0/284: dwrite de/d11/f4c [0,4194304] 0 2024-03-11T19:55:54.133 INFO:tasks.workunit.client.0.smithi107.stdout:8/291: mknod dd/de/d1b/d20/c54 0 2024-03-11T19:55:54.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/296: symlink d4/d1c/l71 0 2024-03-11T19:55:54.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/267: getdents d4/d7/d1f/d2b 0 2024-03-11T19:55:54.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/361: symlink da/l6f 0 2024-03-11T19:55:54.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/362: truncate da/d14/d38/d59/f69 1287933 0 2024-03-11T19:55:54.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/363: chown da/d14/d18/f27 32677043 1 2024-03-11T19:55:54.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/364: write da/d31/d42/f13 [5069543,40689] 0 2024-03-11T19:55:54.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/365: fsync da/d31/d42/f1f 0 2024-03-11T19:55:54.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/285: read de/d11/f4d [67779,64764] 0 2024-03-11T19:55:54.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/292: rename l5 to dd/de/d24/l55 0 2024-03-11T19:55:54.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/297: truncate d4/d1c/f38 1362162 0 2024-03-11T19:55:54.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/268: rmdir d4/d7/d1f/d2b/d19/d4d 0 2024-03-11T19:55:54.142 INFO:tasks.workunit.client.0.smithi107.stdout:5/366: write da/d31/d42/f28 [2552065,13009] 0 2024-03-11T19:55:54.145 INFO:tasks.workunit.client.0.smithi107.stdout:9/298: rename d4/d9/lb to d4/d1d/d1f/d28/d2c/d56/l72 0 2024-03-11T19:55:54.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/269: unlink d4/d7/d2f/d3c/f47 0 2024-03-11T19:55:54.148 INFO:tasks.workunit.client.0.smithi107.stdout:9/299: creat d4/d1d/d1f/d28/d2c/d56/d6d/d17/f73 x:0 0 0 2024-03-11T19:55:54.148 INFO:tasks.workunit.client.0.smithi107.stdout:9/300: stat d4/d1d/d1f 0 2024-03-11T19:55:54.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/270: rmdir d4/d7/d2f/d33 39 2024-03-11T19:55:54.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/271: link d4/d7/d1f/d2b/d19/c1e d4/d7/d1f/d2b/c50 0 2024-03-11T19:55:54.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/272: read f3 [517475,44904] 0 2024-03-11T19:55:54.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/344: dwrite d6/da/d16/f36 [0,4194304] 0 2024-03-11T19:55:54.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/345: getdents d6/d9/d57/d65 0 2024-03-11T19:55:54.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/346: creat d6/d9/d13/f67 x:0 0 0 2024-03-11T19:55:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/301: rename d4/d1c/f38 to d4/d1d/d1f/d28/d2c/d56/d6d/d17/f74 0 2024-03-11T19:55:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/302: read - d4/d1d/d1f/d28/d2c/d56/d6d/d17/f73 zero size 2024-03-11T19:55:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/303: truncate d4/d1d/d1f/d28/d2c/d56/d6d/d17/f73 528077 0 2024-03-11T19:55:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/304: creat d4/d1c/d37/f75 x:0 0 0 2024-03-11T19:55:54.159 INFO:tasks.workunit.client.0.smithi107.stdout:7/347: creat d6/da/d16/d5c/f68 x:0 0 0 2024-03-11T19:55:54.159 INFO:tasks.workunit.client.0.smithi107.stdout:7/348: chown d6/d9/l34 28507715 1 2024-03-11T19:55:54.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/305: mknod d4/d1d/d1f/d28/d2c/c76 0 2024-03-11T19:55:54.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/349: truncate d6/d21/f3a 1435082 0 2024-03-11T19:55:54.167 INFO:tasks.workunit.client.0.smithi107.stdout:6/280: dwrite d10/d16/d24/d2e/f55 [4194304,4194304] 0 2024-03-11T19:55:54.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/306: creat d4/d1d/d1f/d28/d2c/d56/f77 x:0 0 0 2024-03-11T19:55:54.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/281: creat d10/d19/d38/f5c x:0 0 0 2024-03-11T19:55:54.181 INFO:tasks.workunit.client.0.smithi107.stdout:9/307: getdents d4 0 2024-03-11T19:55:54.181 INFO:tasks.workunit.client.0.smithi107.stdout:9/308: getdents d4 0 2024-03-11T19:55:54.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/309: chown d4/d9/d4d 31713 1 2024-03-11T19:55:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:4/273: read d4/d7/d1f/d2b/d24/d26/d29/f3f [2768633,38452] 0 2024-03-11T19:55:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/367: read da/d14/f5e [7374059,65822] 0 2024-03-11T19:55:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/368: dread - da/d14/d38/d59/d54/f57 zero size 2024-03-11T19:55:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:1/304: dwrite d6/d19/d1e/f21 [0,4194304] 0 2024-03-11T19:55:54.185 INFO:tasks.workunit.client.0.smithi107.stdout:1/305: symlink d6/d19/d1e/d27/l5d 0 2024-03-11T19:55:54.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/310: rename d4/d1d/d1f/d28/d2c/d56/d6d/d17 to d4/d1d/d1f/d78 0 2024-03-11T19:55:54.186 INFO:tasks.workunit.client.0.smithi107.stdout:9/311: mkdir d4/d1d/d79 0 2024-03-11T19:55:54.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/274: rename d4/d7/d1f/d2b/f15 to d4/d7/d1f/d2b/d24/d26/f51 0 2024-03-11T19:55:54.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/275: readlink d4/d7/d1f/l40 0 2024-03-11T19:55:54.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/350: dread d6/da/d16/d5c/f4a [0,4194304] 0 2024-03-11T19:55:54.201 INFO:tasks.workunit.client.0.smithi107.stdout:5/369: dread da/d14/d38/d59/f69 [0,4194304] 0 2024-03-11T19:55:54.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/370: write da/d14/d38/d59/d5f/f6a [799673,4040] 0 2024-03-11T19:55:54.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/282: dread f8 [4194304,4194304] 0 2024-03-11T19:55:54.229 INFO:tasks.workunit.client.0.smithi107.stdout:2/276: sync 2024-03-11T19:55:54.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/289: sync 2024-03-11T19:55:54.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/286: dwrite de/d11/f2c [0,4194304] 0 2024-03-11T19:55:54.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/351: dread d6/da/d16/f2e [0,4194304] 0 2024-03-11T19:55:54.239 INFO:tasks.workunit.client.0.smithi107.stdout:6/283: dread d10/d16/d24/d2e/f34 [0,4194304] 0 2024-03-11T19:55:54.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/290: truncate d1/d19/f32 838287 0 2024-03-11T19:55:54.242 INFO:tasks.workunit.client.0.smithi107.stdout:1/306: dwrite d6/d28/f4b [0,4194304] 0 2024-03-11T19:55:54.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/307: write d6/d28/f3f [4728614,31741] 0 2024-03-11T19:55:54.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/287: mkdir de/d58 0 2024-03-11T19:55:54.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/288: chown de/f44 109587 1 2024-03-11T19:55:54.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/289: truncate de/d11/d23/d32/d34/d39/f53 192779 0 2024-03-11T19:55:54.244 INFO:tasks.workunit.client.0.smithi107.stdout:1/308: rename d6/c8 to d6/d19/c5e 0 2024-03-11T19:55:54.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/309: symlink d6/d19/d39/l5f 0 2024-03-11T19:55:54.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/310: fsync d6/f54 0 2024-03-11T19:55:54.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/311: write d6/f55 [472402,129265] 0 2024-03-11T19:55:54.254 INFO:tasks.workunit.client.0.smithi107.stdout:2/277: dread d1/d1a/f2f [0,4194304] 0 2024-03-11T19:55:54.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/276: dwrite d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:55:54.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/277: creat d4/d7/d1f/f52 x:0 0 0 2024-03-11T19:55:54.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/290: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:54.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/291: chown de/f2f 1699975 1 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.271+0000 7fc213200640 1 -- 172.21.15.107:0/1098421499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141062c0 msgr2=0x7fc21410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.271+0000 7fc213200640 1 --2- 172.21.15.107:0/1098421499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141062c0 0x7fc21410de90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc204009890 tx=0x7fc20402bbb0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 -- 172.21.15.107:0/1098421499 shutdown_connections 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 --2- 172.21.15.107:0/1098421499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141062c0 0x7fc21410de90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 --2- 172.21.15.107:0/1098421499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc214101ba0 0x7fc214105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 --2- 172.21.15.107:0/1098421499 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141011f0 0x7fc2141015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 -- 172.21.15.107:0/1098421499 >> 172.21.15.107:0/1098421499 conn(0x7fc2140fca90 msgr2=0x7fc2140feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 -- 172.21.15.107:0/1098421499 shutdown_connections 2024-03-11T19:55:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.272+0000 7fc213200640 1 -- 172.21.15.107:0/1098421499 wait complete. 2024-03-11T19:55:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.273+0000 7fc213200640 1 Processor -- start 2024-03-11T19:55:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 -- start start 2024-03-11T19:55:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 0x7fc21419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 0x7fc21419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21410b300 0x7fc21410b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc211400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 0x7fc21419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21410bf40 con 0x7fc21410b300 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc211e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 0x7fc21419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc211400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 0x7fc21419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46624/0 (socket says 172.21.15.107:46624) 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc211400640 1 -- 172.21.15.107:0/3940583492 learned_addr learned my addr 172.21.15.107:0/3940583492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21410c0b0 con 0x7fc2141011f0 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.274+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2141a1f20 con 0x7fc2141062c0 2024-03-11T19:55:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc212800640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21410b300 0x7fc21410b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 -- 172.21.15.107:0/3940583492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 msgr2=0x7fc21419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 0x7fc21419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 -- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21410b300 msgr2=0x7fc21410b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21410b300 0x7fc21410b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 -- 172.21.15.107:0/3940583492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204009490 con 0x7fc2141011f0 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc211e00640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 0x7fc21419a200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00be20 tx=0x7fc20c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc20aa00640 1 -- 172.21.15.107:0/3940583492 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c014440 con 0x7fc2141011f0 2024-03-11T19:55:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc20aa00640 1 -- 172.21.15.107:0/3940583492 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc20c014a80 con 0x7fc2141011f0 2024-03-11T19:55:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2141a2200 con 0x7fc2141011f0 2024-03-11T19:55:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.275+0000 7fc20aa00640 1 -- 172.21.15.107:0/3940583492 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c012ac0 con 0x7fc2141011f0 2024-03-11T19:55:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.276+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141a2760 con 0x7fc2141011f0 2024-03-11T19:55:54.281 INFO:tasks.workunit.client.0.smithi107.stdout:2/278: dread d1/d1a/f2c [0,4194304] 0 2024-03-11T19:55:54.281 INFO:tasks.workunit.client.0.smithi107.stdout:5/371: dwrite da/d14/d18/f27 [0,4194304] 0 2024-03-11T19:55:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.284+0000 7fc21bbd5740 1 Processor -- start 2024-03-11T19:55:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.284+0000 7fc20aa00640 1 -- 172.21.15.107:0/3940583492 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc20c014bf0 con 0x7fc2141011f0 2024-03-11T19:55:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.284+0000 7fc21bbd5740 1 -- start start 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 0x5646a939fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93a0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a9570 0x5646a93ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a935b620 con 0x5646a939fff0 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a930d8c0 con 0x5646a93a0d60 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a930adf0 con 0x5646a93a9570 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc212800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 0x5646a939fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc212800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 0x5646a939fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38524/0 (socket says 172.21.15.107:38524) 2024-03-11T19:55:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.285+0000 7fc212800640 1 -- 172.21.15.107:0/730706978 learned_addr learned my addr 172.21.15.107:0/730706978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:54.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/352: write d6/da/d16/d5c/f54 [3472000,97649] 0 2024-03-11T19:55:54.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/312: dwrite d4/d1d/d1f/d28/d2c/d56/f77 [0,4194304] 0 2024-03-11T19:55:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.287+0000 7fc211e00640 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.287+0000 7fc20aa00640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 0x7fc1f80a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 -- 172.21.15.107:0/730706978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a9570 msgr2=0x5646a93ab960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:55:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a9570 0x5646a93ab960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 -- 172.21.15.107:0/730706978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 msgr2=0x5646a939fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 0x5646a939fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 -- 172.21.15.107:0/730706978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a9287b00 con 0x5646a939fff0 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.288+0000 7fc211e00640 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93a0470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0caac0 tx=0x7fc20c09a190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f2638f0d06c0b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.289+0000 7fc1ff400640 1 -- 172.21.15.107:0/730706978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0b1070 con 0x5646a939fff0 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.289+0000 7fc1ff400640 1 -- 172.21.15.107:0/730706978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc20c098350 con 0x5646a939fff0 2024-03-11T19:55:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.289+0000 7fc1ff400640 1 -- 172.21.15.107:0/730706978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0d0e00 con 0x5646a939fff0 2024-03-11T19:55:54.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/372: mknod da/d14/d38/d59/d5f/c70 0 2024-03-11T19:55:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.290+0000 7fc20aa00640 1 -- 172.21.15.107:0/3940583492 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc20c0cd680 con 0x7fc2141011f0 2024-03-11T19:55:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.290+0000 7fc21bbd5740 1 -- 172.21.15.107:0/730706978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 msgr2=0x5646a93a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.294 INFO:tasks.workunit.client.0.smithi107.stdout:9/313: truncate f1 2976983 0 2024-03-11T19:55:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.290+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93a0470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0caac0 tx=0x7fc20c09a190 comp rx=0 tx=0).stop 2024-03-11T19:55:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc212800640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 0x7fc1f80a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.295 INFO:tasks.workunit.client.0.smithi107.stdout:5/373: rmdir da 39 2024-03-11T19:55:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 -- 172.21.15.107:0/730706978 shutdown_connections 2024-03-11T19:55:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a9570 0x5646a93ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93a0470 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/730706978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a0d60 0x5646a939fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 -- 172.21.15.107:0/730706978 >> 172.21.15.107:0/730706978 conn(0x5646a93674b0 msgr2=0x5646a939ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 -- 172.21.15.107:0/730706978 shutdown_connections 2024-03-11T19:55:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.291+0000 7fc21bbd5740 1 -- 172.21.15.107:0/730706978 wait complete. 2024-03-11T19:55:54.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/314: unlink d4/d1d/d1f/c6f 0 2024-03-11T19:55:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.293+0000 7fc212800640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 0x7fc1f80a41a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc204037040 tx=0x7fc204005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.293+0000 7fc21bbd5740 1 -- start start 2024-03-11T19:55:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.293+0000 7fc21bbd5740 1 Processor -- start 2024-03-11T19:55:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 0x5646a93a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93a0d60 0x5646a93a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a93065d0 con 0x5646a93a0d60 2024-03-11T19:55:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a92abb50 con 0x5646a939fff0 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a92ab7f0 con 0x5646a93a7170 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc21bbd5740 10 client.0 _ll_register_callbacks cb 0x5646a92bf550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc212800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc212800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:46644/0 (socket says 172.21.15.107:46644) 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.294+0000 7fc212800640 1 -- 172.21.15.107:0/3739630522 learned_addr learned my addr 172.21.15.107:0/3739630522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 -- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 msgr2=0x5646a93a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc211e00640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 0x5646a93a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 0x5646a93a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 -- 172.21.15.107:0/3739630522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93a0d60 msgr2=0x5646a93a6c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93a0d60 0x5646a93a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 -- 172.21.15.107:0/3739630522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141a2760 con 0x5646a93a7170 2024-03-11T19:55:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc211e00640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 0x5646a93a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:55:54.299 INFO:tasks.workunit.client.0.smithi107.stdout:5/374: creat da/f71 x:0 0 0 2024-03-11T19:55:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc212800640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc204034b80 tx=0x7fc20402c0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec94a1d471fe9a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.295+0000 7fc1f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:55:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.296+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204004460 con 0x5646a93a7170 2024-03-11T19:55:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.296+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc20402c550 con 0x5646a93a7170 2024-03-11T19:55:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.296+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646a92d8cd0 con 0x5646a93a7170 2024-03-11T19:55:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.299+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc204005680 con 0x5646a93a7170 2024-03-11T19:55:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.299+0000 7fc21bbd5740 10 client.25714 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.299+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5646a9304c50 con 0x5646a93a7170 2024-03-11T19:55:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.299+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5646a92d8cd0 con 0x5646a93a7170 2024-03-11T19:55:54.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/312: dwrite d6/d19/d1e/d27/f47 [0,4194304] 0 2024-03-11T19:55:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fc20403e040 con 0x5646a93a7170 2024-03-11T19:55:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 10 client.25714 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 20 client.25714 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a93b9bb0 0x5646a93bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 4 client.25714 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.300+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646a92fcce0 con 0x5646a93b9bb0 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.301+0000 7fc211e00640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a93b9bb0 0x5646a93bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.301+0000 7fc211e00640 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a93b9bb0 0x5646a93bbff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a9be51e777b4ed server_cookie=eb190ba546368ffb in_seq=0 out_seq=0 2024-03-11T19:55:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.301+0000 7fc1f3e00640 10 client.25714 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:54.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/313: mkdir d6/d28/d60 0 2024-03-11T19:55:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.302+0000 7fc1f3e00640 1 -- 172.21.15.107:0/3739630522 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646a92fcce0 con 0x5646a93b9bb0 2024-03-11T19:55:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.302+0000 7fc1f3e00640 10 client.25714 handle_command_reply: tid=0 2024-03-11T19:55:54.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/314: rmdir d6/d19/d3d 39 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.303+0000 7fc21bbd5740 1 client.25714 shutdown 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.303+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 msgr2=0x5646a93b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.303+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc204034b80 tx=0x7fc20402c0d0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.303+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 shutdown_connections 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.303+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a93b9bb0 0x5646a93bbff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b5450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93a0d60 0x5646a93a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/3739630522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a939fff0 0x5646a93a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 >> 172.21.15.107:0/3739630522 conn(0x5646a93674b0 msgr2=0x5646a939ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc1f3e00640 0 client.25714 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 shutdown_connections 2024-03-11T19:55:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.304+0000 7fc21bbd5740 1 -- 172.21.15.107:0/3739630522 wait complete. 2024-03-11T19:55:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.305+0000 7fc21bbd5740 20 client.25714 trim_cache size 0 max 16384 2024-03-11T19:55:54.308 INFO:tasks.workunit.client.0.smithi107.stdout:9/315: dread d4/f54 [0,4194304] 0 2024-03-11T19:55:54.308 INFO:tasks.workunit.client.0.smithi107.stdout:1/315: rename d6/d19/d1e/l4f to d6/d5b/l61 0 2024-03-11T19:55:54.309 INFO:tasks.workunit.client.0.smithi107.stdout:1/316: write d6/d19/d1e/f4e [87041,16636] 0 2024-03-11T19:55:54.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/317: readlink d6/d19/d1e/d27/d2c/l42 0 2024-03-11T19:55:54.310 INFO:tasks.workunit.client.0.smithi107.stdout:9/316: creat d4/d1d/d79/f7a x:0 0 0 2024-03-11T19:55:54.310 INFO:tasks.workunit.client.0.smithi107.stdout:9/317: chown d4/d1d/d1f/l36 20029294 1 2024-03-11T19:55:54.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/318: rmdir d6/d19/d1e/d27/d2c/d32 39 2024-03-11T19:55:54.312 INFO:tasks.workunit.client.0.smithi107.stdout:9/318: link d4/d1d/d1f/d28/d2c/d56/d6d/c25 d4/d1d/d1f/d28/d2c/c7b 0 2024-03-11T19:55:54.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/319: rename d4/d1d/d1f/d28/d2c/d56/d6d/c25 to d4/d9/c7c 0 2024-03-11T19:55:54.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/279: dread d1/f10 [4194304,4194304] 0 2024-03-11T19:55:54.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/280: chown d1/d1a/d1e/d26/f32 0 1 2024-03-11T19:55:54.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/278: dwrite d4/f6 [0,4194304] 0 2024-03-11T19:55:54.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/279: chown d4/d7/d1f/l40 2 1 2024-03-11T19:55:54.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/280: rmdir d4/d7/d1f 39 2024-03-11T19:55:54.334 INFO:tasks.workunit.client.0.smithi107.stdout:1/319: write d6/f1b [2730274,65785] 0 2024-03-11T19:55:54.335 INFO:tasks.workunit.client.0.smithi107.stdout:4/281: link d4/d7/d1f/d2b/d19/c1e d4/d7/c53 0 2024-03-11T19:55:54.335 INFO:tasks.workunit.client.0.smithi107.stdout:4/282: chown d4/d7/c53 268 1 2024-03-11T19:55:54.336 INFO:tasks.workunit.client.0.smithi107.stdout:4/283: mknod d4/d7/d2f/d33/c54 0 2024-03-11T19:55:54.337 INFO:tasks.workunit.client.0.smithi107.stdout:4/284: symlink d4/d7/d2f/d33/d3a/l55 0 2024-03-11T19:55:54.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/285: fsync d4/d7/db/f21 0 2024-03-11T19:55:54.339 INFO:tasks.workunit.client.0.smithi107.stdout:4/286: creat d4/d7/d1f/d2b/f56 x:0 0 0 2024-03-11T19:55:54.340 INFO:tasks.workunit.client.0.smithi107.stdout:5/375: write da/d31/d42/f23 [3011080,90231] 0 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 Processor -- start 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 -- start start 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a0d60 0x5646a93b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a904ee60 con 0x5646a939fff0 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a93963d0 con 0x5646a93a7170 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.332+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a939bce0 con 0x5646a93a0d60 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38544/0 (socket says 172.21.15.107:38544) 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 -- 172.21.15.107:0/2956022277 learned_addr learned my addr 172.21.15.107:0/2956022277 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 -- 172.21.15.107:0/2956022277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a0d60 msgr2=0x5646a93b08d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a0d60 0x5646a93b08d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 -- 172.21.15.107:0/2956022277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 msgr2=0x5646a93b0390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93b0390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 -- 172.21.15.107:0/2956022277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a92a1530 con 0x5646a93a7170 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.333+0000 7fc212800640 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc20402c1c0 tx=0x7fc204005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67967063ed39847 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.334+0000 7fc1fea00640 1 -- 172.21.15.107:0/2956022277 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204072070 con 0x5646a93a7170 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.334+0000 7fc1fea00640 1 -- 172.21.15.107:0/2956022277 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc204003ed0 con 0x5646a93a7170 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.334+0000 7fc1fea00640 1 -- 172.21.15.107:0/2956022277 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040049b0 con 0x5646a93a7170 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 -- 172.21.15.107:0/2956022277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 msgr2=0x5646a939a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc20402c1c0 tx=0x7fc204005700 comp rx=0 tx=0).stop 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 -- 172.21.15.107:0/2956022277 shutdown_connections 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93a7170 0x5646a939a810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a0d60 0x5646a93b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/2956022277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a939fff0 0x5646a93b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.335+0000 7fc21bbd5740 1 -- 172.21.15.107:0/2956022277 >> 172.21.15.107:0/2956022277 conn(0x5646a93674b0 msgr2=0x5646a93b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/291: dwrite d1/fa [0,4194304] 0 2024-03-11T19:55:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.341+0000 7fc21bbd5740 1 -- 172.21.15.107:0/2956022277 shutdown_connections 2024-03-11T19:55:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.341+0000 7fc21bbd5740 1 -- 172.21.15.107:0/2956022277 wait complete. 2024-03-11T19:55:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.342+0000 7fc21bbd5740 1 -- start start 2024-03-11T19:55:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.343+0000 7fc21bbd5740 1 Processor -- start 2024-03-11T19:55:54.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/376: truncate da/d14/d18/f51 2222891 0 2024-03-11T19:55:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.344+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.344+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 0x5646a93ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc21bbd5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 0x5646a940f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a940f6e0 con 0x5646a93ace90 2024-03-11T19:55:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a940f820 con 0x5646a93ac370 2024-03-11T19:55:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc21bbd5740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a940f960 con 0x5646a93a7170 2024-03-11T19:55:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc21bbd5740 10 client.0 _ll_register_callbacks cb 0x7fc204043040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc211400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 0x5646a93ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc211400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 0x5646a93ac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38554/0 (socket says 172.21.15.107:38554) 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.345+0000 7fc211400640 1 -- 172.21.15.107:0/1635233194 learned_addr learned my addr 172.21.15.107:0/1635233194 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 0x5646a940f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc211e00640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 -- 172.21.15.107:0/1635233194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 msgr2=0x5646a93b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 -- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 msgr2=0x5646a93ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 0x5646a93ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.346+0000 7fc212800640 1 -- 172.21.15.107:0/1635233194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646a9304c50 con 0x5646a93ace90 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.347+0000 7fc211e00640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.347+0000 7fc212800640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 0x5646a940f100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc204005010 tx=0x7fc204004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d31d516b099556 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.347+0000 7fc1f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204072040 con 0x5646a93ace90 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc204004260 con 0x5646a93ace90 2024-03-11T19:55:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646a9006600 con 0x5646a93ace90 2024-03-11T19:55:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc204003bb0 con 0x5646a93ace90 2024-03-11T19:55:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc21bbd5740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-03-11T19:55:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.348+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5646a92c72c0 con 0x5646a93ace90 2024-03-11T19:55:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.350+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc204083030 con 0x5646a93ace90 2024-03-11T19:55:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.350+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fc20402ca10 con 0x5646a93ace90 2024-03-11T19:55:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.350+0000 7fc21bbd5740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-03-11T19:55:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.350+0000 7fc21bbd5740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:55:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.350+0000 7fc21bbd5740 20 client.15585 populate_metadata read hostname 'smithi107' 2024-03-11T19:55:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.351+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a9410ce0 0x5646a94130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.351+0000 7fc21bbd5740 4 client.15585 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:55:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.351+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646a90571b0 con 0x5646a9410ce0 2024-03-11T19:55:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.352+0000 7fc211e00640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a9410ce0 0x5646a94130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:54.359 INFO:tasks.workunit.client.0.smithi107.stdout:5/377: mkdir da/d14/d38/d59/d54/d72 0 2024-03-11T19:55:54.359 INFO:tasks.workunit.client.0.smithi107.stdout:5/378: fsync f5 0 2024-03-11T19:55:54.360 INFO:tasks.workunit.client.0.smithi107.stdout:2/281: dread d1/f49 [0,4194304] 0 2024-03-11T19:55:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.358+0000 7fc211e00640 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a9410ce0 0x5646a94130d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c70e3701d04bf42 server_cookie=1497158dc3b43bd2 in_seq=0 out_seq=0 2024-03-11T19:55:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.358+0000 7fc1f3e00640 10 client.15585 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:54.368 INFO:tasks.workunit.client.0.smithi107.stdout:5/379: mkdir da/d14/d38/d59/d54/d73 0 2024-03-11T19:55:54.368 INFO:tasks.workunit.client.0.smithi107.stdout:5/380: truncate da/d14/d18/d1e/f65 576693 0 2024-03-11T19:55:54.369 INFO:tasks.workunit.client.0.smithi107.stdout:2/282: mknod d1/d1a/d1e/c59 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/293: sync 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/294: fdatasync dd/d13/f3d 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/320: dread d6/d19/f43 [0,4194304] 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/320: dwrite d4/d1d/f3b [0,4194304] 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/321: creat d4/d1d/d1f/d28/d2c/d56/f7d x:0 0 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/322: fdatasync d4/d1d/d1f/f39 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/353: fdatasync d6/da/d16/d5c/f54 0 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/354: chown d6/d9/l64 2 1 2024-03-11T19:55:54.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/355: chown d6/d21 14 1 2024-03-11T19:55:54.385 INFO:tasks.workunit.client.0.smithi107.stdout:0/292: dwrite de/d11/d23/d24/f4a [0,4194304] 0 2024-03-11T19:55:54.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/287: dread d4/fc [0,4194304] 0 2024-03-11T19:55:54.385 INFO:tasks.workunit.client.0.smithi107.stdout:0/293: fdatasync de/f18 0 2024-03-11T19:55:54.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/288: chown d4/d7/d1f/l39 2360 1 2024-03-11T19:55:54.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/289: dread d4/d7/d2f/d33/d3a/f44 [0,4194304] 0 2024-03-11T19:55:54.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/290: write d4/d7/db/f21 [216162,107516] 0 2024-03-11T19:55:54.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/291: readlink d4/d7/d1f/l25 0 2024-03-11T19:55:54.400 INFO:tasks.workunit.client.0.smithi107.stdout:3/292: dwrite d1/d10/d13/d4a/d4c/d1a/f41 [0,4194304] 0 2024-03-11T19:55:54.402 INFO:tasks.workunit.client.0.smithi107.stdout:1/321: dread d6/f1b [0,4194304] 0 2024-03-11T19:55:54.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/322: readlink d6/d19/d1e/l52 0 2024-03-11T19:55:54.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/295: dwrite dd/f4c [0,4194304] 0 2024-03-11T19:55:54.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/292: write d4/d7/f1d [1230168,76435] 0 2024-03-11T19:55:54.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/381: truncate da/d14/d18/f2c 1749752 0 2024-03-11T19:55:54.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/382: write da/f32 [3668162,126498] 0 2024-03-11T19:55:54.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/383: write da/d14/d38/f3f [953019,102148] 0 2024-03-11T19:55:54.449 INFO:tasks.workunit.client.0.smithi107.stdout:7/356: creat d6/d9/d13/d60/d61/f69 x:0 0 0 2024-03-11T19:55:54.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/294: getdents de/d11/d23/d24 0 2024-03-11T19:55:54.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/295: chown de/d11/d23/d32/d34/d39/d55 4825035 1 2024-03-11T19:55:54.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/296: chown de/f3d 158757235 1 2024-03-11T19:55:54.458 INFO:tasks.workunit.client.0.smithi107.stdout:3/293: mkdir d1/d10/d21/d43/d4f 0 2024-03-11T19:55:54.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/296: mkdir dd/de/d1b/d20/d56 0 2024-03-11T19:55:54.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/293: getdents d4/d7/d1f/d2b/d24/d26/d29 0 2024-03-11T19:55:54.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/323: dwrite d6/f1b [4194304,4194304] 0 2024-03-11T19:55:54.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/324: mkdir d6/d19/d3d/d62 0 2024-03-11T19:55:54.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/325: creat d6/d28/d60/f63 x:0 0 0 2024-03-11T19:55:54.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/326: rmdir d6/d19/d1e/d27/d2c 39 2024-03-11T19:55:54.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/327: link d6/d19/d1e/d27/l3b d6/d19/d1e/d27/d2c/d32/d58/l64 0 2024-03-11T19:55:54.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/294: creat d1/d10/d21/d43/d48/f50 x:0 0 0 2024-03-11T19:55:54.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/297: mknod dd/de/d1b/d20/c57 0 2024-03-11T19:55:54.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/298: getdents dd/de/d49/d46 0 2024-03-11T19:55:54.610 INFO:tasks.workunit.client.0.smithi107.stdout:7/357: dread d6/da/d16/f36 [0,4194304] 0 2024-03-11T19:55:54.610 INFO:tasks.workunit.client.0.smithi107.stdout:7/358: write d6/d21/f26 [931881,125634] 0 2024-03-11T19:55:54.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/295: dwrite d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:55:54.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/328: dwrite d6/d19/d1e/d27/f3e [0,4194304] 0 2024-03-11T19:55:54.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/329: fsync d6/d19/d1e/d27/d2c/f34 0 2024-03-11T19:55:54.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/330: stat d6/d19/d3d/d62 0 2024-03-11T19:55:54.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/331: chown d6/d19/d1e/d27/d2c/l42 3 1 2024-03-11T19:55:54.633 INFO:tasks.workunit.client.0.smithi107.stdout:1/332: read d6/d19/d1e/d27/d2c/f30 [1352054,102833] 0 2024-03-11T19:55:54.647 INFO:tasks.workunit.client.0.smithi107.stdout:3/296: write d1/d10/d2a/f38 [674715,66774] 0 2024-03-11T19:55:54.647 INFO:tasks.workunit.client.0.smithi107.stdout:3/297: write d1/d19/d2d/f3c [120002,120307] 0 2024-03-11T19:55:54.652 INFO:tasks.workunit.client.0.smithi107.stdout:8/299: dwrite dd/de/d49/f42 [0,4194304] 0 2024-03-11T19:55:54.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/333: write d6/d19/d1e/d27/f47 [2655126,103536] 0 2024-03-11T19:55:54.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/334: getdents d6/d5b 0 2024-03-11T19:55:54.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/335: write d6/d19/d1e/f29 [4597788,25526] 0 2024-03-11T19:55:54.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/336: write d6/f55 [1377055,97022] 0 2024-03-11T19:55:54.655 INFO:tasks.workunit.client.0.smithi107.stdout:7/359: dwrite d6/f17 [0,4194304] 0 2024-03-11T19:55:54.655 INFO:tasks.workunit.client.0.smithi107.stdout:1/337: creat d6/d19/d3d/d62/f65 x:0 0 0 2024-03-11T19:55:54.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/338: mknod d6/d19/d39/c66 0 2024-03-11T19:55:54.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/339: chown d6/d19/d1e/l1f 24 1 2024-03-11T19:55:54.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/340: mkdir d6/d19/d1e/d27/d2c/d67 0 2024-03-11T19:55:54.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/341: stat d6/d28/d60/f63 0 2024-03-11T19:55:54.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/298: dread d1/fa [0,4194304] 0 2024-03-11T19:55:54.670 INFO:tasks.workunit.client.0.smithi107.stdout:5/384: symlink da/d14/d18/l74 0 2024-03-11T19:55:54.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/300: dread dd/de/d49/f42 [0,4194304] 0 2024-03-11T19:55:54.674 INFO:tasks.workunit.client.0.smithi107.stdout:3/299: getdents d1/d19 0 2024-03-11T19:55:54.674 INFO:tasks.workunit.client.0.smithi107.stdout:3/300: dread - d1/d19/f44 zero size 2024-03-11T19:55:54.674 INFO:tasks.workunit.client.0.smithi107.stdout:3/301: truncate d1/d19/f44 355509 0 2024-03-11T19:55:54.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/323: rmdir d4/d1d/d1f/d28/d2c/d56/d6d 39 2024-03-11T19:55:54.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/283: rename d1/d38/l3a to d1/l5a 0 2024-03-11T19:55:54.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/324: creat d4/d1c/f7e x:0 0 0 2024-03-11T19:55:54.689 INFO:tasks.workunit.client.0.smithi107.stdout:4/294: rename d4/d7/d1f/d2b/d24/d26/d29/f2d to d4/d7/d1f/d2b/d24/d26/d29/d36/f57 0 2024-03-11T19:55:54.691 INFO:tasks.workunit.client.0.smithi107.stdout:9/325: creat d4/d1d/d1f/d28/d5d/f7f x:0 0 0 2024-03-11T19:55:54.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/302: write d1/d10/d26/f25 [1030303,64212] 0 2024-03-11T19:55:54.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/360: rename d6/d9/d13/d60/d61/f69 to d6/d9/d57/d65/f6a 0 2024-03-11T19:55:54.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/342: dwrite d6/d19/d1e/d27/d2c/f30 [0,4194304] 0 2024-03-11T19:55:54.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/295: truncate d4/f6 362514 0 2024-03-11T19:55:54.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/343: creat d6/f68 x:0 0 0 2024-03-11T19:55:54.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/344: stat d6/d19/d1e/d27/d2c 0 2024-03-11T19:55:54.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/303: symlink d1/d10/d26/l51 0 2024-03-11T19:55:54.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/304: stat d1/d10/d26/f30 0 2024-03-11T19:55:54.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/305: write d1/d10/d13/d4a/d4c/d1a/f41 [4506430,85716] 0 2024-03-11T19:55:54.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/296: link d4/d7/d1f/d2b/d24/d26/d29/f3f d4/d7/d2f/d3b/f58 0 2024-03-11T19:55:54.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/297: dread - d4/d7/d1f/d2b/d24/f3e zero size 2024-03-11T19:55:54.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/284: dread d1/fc [0,4194304] 0 2024-03-11T19:55:54.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/306: mknod d1/d10/d2a/c52 0 2024-03-11T19:55:54.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/297: creat de/f59 x:0 0 0 2024-03-11T19:55:54.710 INFO:tasks.workunit.client.0.smithi107.stdout:6/284: sync 2024-03-11T19:55:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.712+0000 7fc1f3e00640 1 -- 172.21.15.107:0/1635233194 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646a90571b0 con 0x5646a9410ce0 2024-03-11T19:55:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.712+0000 7fc1f3e00640 10 client.15585 handle_command_reply: tid=0 2024-03-11T19:55:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.713+0000 7fc21bbd5740 1 client.15585 shutdown 2024-03-11T19:55:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 msgr2=0x5646a940f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 0x5646a940f100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc204005010 tx=0x7fc204004450 comp rx=0 tx=0).stop 2024-03-11T19:55:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 shutdown_connections 2024-03-11T19:55:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5646a9410ce0 0x5646a94130d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5646a93ac370 0x5646a93ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646a93ace90 0x5646a940f100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 --2- 172.21.15.107:0/1635233194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5646a93a7170 0x5646a93b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc1f3e00640 0 client.15585 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:55:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.715+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 >> 172.21.15.107:0/1635233194 conn(0x5646a93674b0 msgr2=0x5646a939a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/298: readlink de/d11/d23/d32/d34/d39/l52 0 2024-03-11T19:55:54.724 INFO:tasks.workunit.client.0.smithi107.stdout:6/285: write d10/d16/f1e [5487114,47896] 0 2024-03-11T19:55:54.725 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.723+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 shutdown_connections 2024-03-11T19:55:54.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.723+0000 7fc21bbd5740 1 -- 172.21.15.107:0/1635233194 wait complete. 2024-03-11T19:55:54.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/385: dwrite da/d14/d38/d59/f69 [0,4194304] 0 2024-03-11T19:55:54.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/301: dread dd/de/d1b/d2c/f2d [0,4194304] 0 2024-03-11T19:55:54.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/302: dread - dd/de/d49/d46/f4f zero size 2024-03-11T19:55:54.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/303: chown dd/de/d24/d28/d47 290527175 1 2024-03-11T19:55:54.728 INFO:tasks.workunit.client.0.smithi107.stdout:6/286: getdents d10/d16/d24 0 2024-03-11T19:55:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.726+0000 7fc21bbd5740 20 client.15585 trim_cache size 0 max 16384 2024-03-11T19:55:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:55:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:55:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:55:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:55:54.731 INFO:tasks.workunit.client.0.smithi107.stdout:8/304: creat dd/f58 x:0 0 0 2024-03-11T19:55:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.728+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 msgr2=0x7fc1f80a41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.728+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 0x7fc1f80a41a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc204037040 tx=0x7fc204005f90 comp rx=0 tx=0).stop 2024-03-11T19:55:54.732 INFO:tasks.workunit.client.0.smithi107.stdout:5/386: truncate da/d31/d42/f23 2906316 0 2024-03-11T19:55:54.732 INFO:tasks.workunit.client.0.smithi107.stdout:5/387: dread - da/d14/d43/f6c zero size 2024-03-11T19:55:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.729+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 msgr2=0x7fc21419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.729+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 0x7fc21419a200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00be20 tx=0x7fc20c009f70 comp rx=0 tx=0).stop 2024-03-11T19:55:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 shutdown_connections 2024-03-11T19:55:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc1f80a1ce0 0x7fc1f80a41a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21410b300 0x7fc21410b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc2141062c0 0x7fc21419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 --2- 172.21.15.107:0/3940583492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2141011f0 0x7fc21419a200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.730+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 >> 172.21.15.107:0/3940583492 conn(0x7fc2140fca90 msgr2=0x7fc2140fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.731+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 shutdown_connections 2024-03-11T19:55:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:55:54.731+0000 7fc213200640 1 -- 172.21.15.107:0/3940583492 wait complete. 2024-03-11T19:55:54.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/305: mknod dd/de/d24/c59 0 2024-03-11T19:55:54.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/388: getdents da/d14/d43 0 2024-03-11T19:55:54.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/389: creat da/d31/f75 x:0 0 0 2024-03-11T19:55:54.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/306: mknod dd/de/d24/c5a 0 2024-03-11T19:55:54.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/307: creat dd/de/d49/f5b x:0 0 0 2024-03-11T19:55:54.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/308: stat dd/d13/d32 0 2024-03-11T19:55:54.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/309: creat dd/d13/d18/f5c x:0 0 0 2024-03-11T19:55:54.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/299: link de/f17 de/d11/d23/d32/d34/d39/f5a 0 2024-03-11T19:55:54.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/361: dwrite d6/d21/f26 [0,4194304] 0 2024-03-11T19:55:54.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/285: dwrite d1/d1a/f2c [0,4194304] 0 2024-03-11T19:55:54.757 INFO:tasks.workunit.client.0.smithi107.stdout:7/362: rename d6/d21/f26 to d6/d9/f6b 0 2024-03-11T19:55:54.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/300: truncate de/f2f 4126278 0 2024-03-11T19:55:54.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/345: dwrite d6/f55 [0,4194304] 0 2024-03-11T19:55:54.778 INFO:tasks.workunit.client.0.smithi107.stdout:6/287: dread fc [0,4194304] 0 2024-03-11T19:55:54.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/346: truncate d6/f15 966927 0 2024-03-11T19:55:54.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/326: dwrite d4/fe [0,4194304] 0 2024-03-11T19:55:54.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/347: symlink d6/d19/d1e/l69 0 2024-03-11T19:55:54.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/348: chown d6/d28 797574 1 2024-03-11T19:55:54.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/310: write dd/de/d49/f42 [1910114,99409] 0 2024-03-11T19:55:54.785 INFO:tasks.workunit.client.0.smithi107.stdout:9/327: truncate d4/f54 263830 0 2024-03-11T19:55:54.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/311: symlink dd/d13/d32/d51/l5d 0 2024-03-11T19:55:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/286: dread d1/d21/d35/f54 [0,4194304] 0 2024-03-11T19:55:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/287: creat d1/d1d/f5b x:0 0 0 2024-03-11T19:55:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/288: write d1/d1a/d1e/d26/f48 [827935,41267] 0 2024-03-11T19:55:54.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/298: dwrite d4/f6 [0,4194304] 0 2024-03-11T19:55:54.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/299: chown d4/d7/c53 239395485 1 2024-03-11T19:55:54.805 INFO:tasks.workunit.client.0.smithi107.stdout:2/289: creat d1/d21/d35/f5c x:0 0 0 2024-03-11T19:55:54.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/290: creat d1/d1a/d34/d4f/f5d x:0 0 0 2024-03-11T19:55:54.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/291: write d1/d1a/d1e/d41/f51 [264432,61801] 0 2024-03-11T19:55:54.810 INFO:tasks.workunit.client.0.smithi107.stdout:2/292: truncate d1/d21/d35/f54 1040919 0 2024-03-11T19:55:54.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/301: dread de/d11/d23/d32/d34/d39/f5a [0,4194304] 0 2024-03-11T19:55:54.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/302: readlink de/l1c 0 2024-03-11T19:55:54.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/303: truncate de/d11/f35 1838137 0 2024-03-11T19:55:54.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/390: dwrite f4 [0,4194304] 0 2024-03-11T19:55:54.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/304: dread de/f21 [0,4194304] 0 2024-03-11T19:55:54.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/305: readlink l1 0 2024-03-11T19:55:54.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/306: write de/f13 [70213,49134] 0 2024-03-11T19:55:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:5/391: chown da/d31/d42/f28 579003215 1 2024-03-11T19:55:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:5/392: readlink da/d14/d18/d1e/l2a 0 2024-03-11T19:55:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:5/393: readlink da/d14/l5c 0 2024-03-11T19:55:54.830 INFO:tasks.workunit.client.0.smithi107.stdout:0/307: creat de/d11/d23/d32/d34/d39/f5b x:0 0 0 2024-03-11T19:55:54.830 INFO:tasks.workunit.client.0.smithi107.stdout:0/308: fdatasync fd 0 2024-03-11T19:55:54.833 INFO:tasks.workunit.client.0.smithi107.stdout:7/363: dread d6/f2f [0,4194304] 0 2024-03-11T19:55:54.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/307: dwrite d1/d10/d13/d4a/d4c/d1a/f41 [0,4194304] 0 2024-03-11T19:55:54.843 INFO:tasks.workunit.client.0.smithi107.stdout:3/308: write d1/d19/d2d/f3c [965195,53334] 0 2024-03-11T19:55:54.843 INFO:tasks.workunit.client.0.smithi107.stdout:3/309: rmdir d1/d10/d13/d4a/d4c/d1a 39 2024-03-11T19:55:54.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/310: unlink d1/d10/d26/f30 0 2024-03-11T19:55:54.850 INFO:tasks.workunit.client.0.smithi107.stdout:3/311: unlink d1/l4 0 2024-03-11T19:55:54.850 INFO:tasks.workunit.client.0.smithi107.stdout:2/293: dread d1/f15 [0,4194304] 0 2024-03-11T19:55:54.864 INFO:tasks.workunit.client.0.smithi107.stdout:4/300: dwrite d4/d7/d1f/d2b/d24/d26/f43 [0,4194304] 0 2024-03-11T19:55:54.880 INFO:tasks.workunit.client.0.smithi107.stdout:7/364: dwrite d6/d9/f6b [0,4194304] 0 2024-03-11T19:55:54.880 INFO:tasks.workunit.client.0.smithi107.stdout:7/365: mknod d6/da/d16/d5c/c6c 0 2024-03-11T19:55:54.889 INFO:tasks.workunit.client.0.smithi107.stdout:6/288: dwrite d10/d19/f31 [0,4194304] 0 2024-03-11T19:55:54.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/289: creat d10/d19/f5d x:0 0 0 2024-03-11T19:55:54.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/290: creat d10/d19/d3b/f5e x:0 0 0 2024-03-11T19:55:54.894 INFO:tasks.workunit.client.0.smithi107.stdout:3/312: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:54.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/291: getdents d10/d19/d38/d40 0 2024-03-11T19:55:54.901 INFO:tasks.workunit.client.0.smithi107.stdout:3/313: chown d1/d10/d13/d4a/d4c/d1a/f20 856 1 2024-03-11T19:55:54.910 INFO:tasks.workunit.client.0.smithi107.stdout:3/314: creat d1/d19/d2d/d4d/f53 x:0 0 0 2024-03-11T19:55:54.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/292: symlink d10/d16/d2b/d39/l5f 0 2024-03-11T19:55:54.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/293: dread - d10/d16/d24/f5b zero size 2024-03-11T19:55:54.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/301: dwrite d4/d7/d1f/d2b/d24/d26/f43 [0,4194304] 0 2024-03-11T19:55:54.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/302: symlink d4/d7/l59 0 2024-03-11T19:55:54.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/303: read d4/d7/f1d [1202338,109242] 0 2024-03-11T19:55:54.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/304: fdatasync d4/d7/d1f/d2b/d24/d26/f51 0 2024-03-11T19:55:54.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/305: mknod d4/d7/d2f/d3c/c5a 0 2024-03-11T19:55:54.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/306: chown d4/d7/db 5330 1 2024-03-11T19:55:54.920 INFO:tasks.workunit.client.0.smithi107.stdout:8/312: write dd/de/f3b [2985566,50894] 0 2024-03-11T19:55:54.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/394: dwrite da/d14/d18/d1e/f65 [0,4194304] 0 2024-03-11T19:55:54.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/315: dread d1/d19/f44 [0,4194304] 0 2024-03-11T19:55:54.926 INFO:tasks.workunit.client.0.smithi107.stdout:1/349: dwrite d6/d19/d1e/f21 [0,4194304] 0 2024-03-11T19:55:54.926 INFO:tasks.workunit.client.0.smithi107.stdout:1/350: write d6/d19/d1e/f4d [1035591,26893] 0 2024-03-11T19:55:54.933 INFO:tasks.workunit.client.0.smithi107.stdout:2/294: dread d1/d1a/f2f [0,4194304] 0 2024-03-11T19:55:54.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/295: chown d1/d1d/c1f 677 1 2024-03-11T19:55:54.939 INFO:tasks.workunit.client.0.smithi107.stdout:8/313: read dd/de/d1b/d20/f29 [1166973,68163] 0 2024-03-11T19:55:54.940 INFO:tasks.workunit.client.0.smithi107.stdout:8/314: truncate dd/de/d24/d28/f4a 629021 0 2024-03-11T19:55:54.940 INFO:tasks.workunit.client.0.smithi107.stdout:8/315: chown dd/f58 29 1 2024-03-11T19:55:54.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/296: truncate d1/f15 108164 0 2024-03-11T19:55:54.943 INFO:tasks.workunit.client.0.smithi107.stdout:8/316: mkdir dd/de/d1b/d20/d5e 0 2024-03-11T19:55:54.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/297: mknod d1/d1a/c5e 0 2024-03-11T19:55:54.949 INFO:tasks.workunit.client.0.smithi107.stdout:2/298: read d1/d1a/f20 [12085,58483] 0 2024-03-11T19:55:54.949 INFO:tasks.workunit.client.0.smithi107.stdout:5/395: dread f5 [0,4194304] 0 2024-03-11T19:55:54.949 INFO:tasks.workunit.client.0.smithi107.stdout:5/396: fdatasync da/d14/d38/d46/f61 0 2024-03-11T19:55:54.949 INFO:tasks.workunit.client.0.smithi107.stdout:5/397: fsync da/d31/f75 0 2024-03-11T19:55:54.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/317: mknod dd/de/d1b/d2c/c5f 0 2024-03-11T19:55:54.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/366: dread d6/f2f [0,4194304] 0 2024-03-11T19:55:54.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/367: truncate d6/d9/f30 93832 0 2024-03-11T19:55:54.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/368: creat d6/d9/f6d x:0 0 0 2024-03-11T19:55:54.964 INFO:tasks.workunit.client.0.smithi107.stdout:7/369: mknod d6/d9/d57/c6e 0 2024-03-11T19:55:54.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/309: dwrite de/d11/d23/d24/f2d [0,4194304] 0 2024-03-11T19:55:54.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/310: fsync de/d11/d23/d32/d34/d39/f5b 0 2024-03-11T19:55:54.965 INFO:tasks.workunit.client.0.smithi107.stdout:6/294: dread d10/d16/f20 [0,4194304] 0 2024-03-11T19:55:54.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/370: creat d6/da/d16/f6f x:0 0 0 2024-03-11T19:55:54.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/371: creat d6/da/d16/d5c/f70 x:0 0 0 2024-03-11T19:55:54.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/351: dread d6/d19/d1e/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:54.975 INFO:tasks.workunit.client.0.smithi107.stdout:4/307: dwrite d4/d7/d1f/d2b/d24/f3e [0,4194304] 0 2024-03-11T19:55:54.982 INFO:tasks.workunit.client.0.smithi107.stdout:5/398: write da/d31/d42/f1f [3763508,130586] 0 2024-03-11T19:55:54.982 INFO:tasks.workunit.client.0.smithi107.stdout:3/316: dwrite d1/d10/d13/f3b [0,4194304] 0 2024-03-11T19:55:54.982 INFO:tasks.workunit.client.0.smithi107.stdout:3/317: dread - d1/d19/d2d/d4d/f53 zero size 2024-03-11T19:55:54.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/372: creat d6/da/d4d/f71 x:0 0 0 2024-03-11T19:55:54.984 INFO:tasks.workunit.client.0.smithi107.stdout:8/318: read dd/d13/d18/f1a [7484994,6747] 0 2024-03-11T19:55:54.986 INFO:tasks.workunit.client.0.smithi107.stdout:5/399: creat da/d14/d18/f76 x:0 0 0 2024-03-11T19:55:54.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/400: read da/d14/f5e [6990233,129759] 0 2024-03-11T19:55:54.989 INFO:tasks.workunit.client.0.smithi107.stdout:3/318: creat d1/d10/d13/d4a/d4c/d49/f54 x:0 0 0 2024-03-11T19:55:54.990 INFO:tasks.workunit.client.0.smithi107.stdout:3/319: write d1/d10/d2a/f38 [2881996,93453] 0 2024-03-11T19:55:54.991 INFO:tasks.workunit.client.0.smithi107.stdout:3/320: stat d1/d10/d26 0 2024-03-11T19:55:54.992 INFO:tasks.workunit.client.0.smithi107.stdout:1/352: creat d6/d19/d1e/d27/d2c/f6a x:0 0 0 2024-03-11T19:55:55.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/353: link d6/d19/d1e/d27/d2c/d32/l3a d6/d19/d1e/d27/l6b 0 2024-03-11T19:55:55.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/308: unlink d4/d7/d2f/d33/d3a/l4a 0 2024-03-11T19:55:55.008 INFO:tasks.workunit.client.0.smithi107.stdout:4/309: creat d4/d7/d1f/d2b/d24/f5b x:0 0 0 2024-03-11T19:55:55.009 INFO:tasks.workunit.client.0.smithi107.stdout:4/310: mkdir d4/d7/db/d5c 0 2024-03-11T19:55:55.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:54 smithi122 ceph-mon[36397]: pgmap v77: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 356 op/s 2024-03-11T19:55:55.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/311: fdatasync d4/d7/d2f/d3b/f4e 0 2024-03-11T19:55:55.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/312: stat d4/d7/d2f/d33/c54 0 2024-03-11T19:55:55.012 INFO:tasks.workunit.client.0.smithi107.stdout:2/299: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:55.024 INFO:tasks.workunit.client.0.smithi107.stdout:6/295: dwrite d10/d19/f23 [0,4194304] 0 2024-03-11T19:55:55.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/296: mkdir d10/d19/d38/d60 0 2024-03-11T19:55:55.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/297: readlink d10/d19/d38/d40/l4f 0 2024-03-11T19:55:55.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:54 smithi145 ceph-mon[36730]: pgmap v77: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 356 op/s 2024-03-11T19:55:55.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/298: write d10/d16/d24/d2e/f2f [67167,90017] 0 2024-03-11T19:55:55.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:54 smithi107 ceph-mon[31712]: pgmap v77: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 356 op/s 2024-03-11T19:55:55.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/313: dwrite d4/d7/f17 [0,4194304] 0 2024-03-11T19:55:55.048 INFO:tasks.workunit.client.0.smithi107.stdout:5/401: dwrite da/f71 [0,4194304] 0 2024-03-11T19:55:55.050 INFO:tasks.workunit.client.0.smithi107.stdout:0/311: dwrite de/d11/f4d [0,4194304] 0 2024-03-11T19:55:55.052 INFO:tasks.workunit.client.0.smithi107.stdout:5/402: mkdir da/d14/d77 0 2024-03-11T19:55:55.052 INFO:tasks.workunit.client.0.smithi107.stdout:5/403: chown da/d14/d18/c55 1245 1 2024-03-11T19:55:55.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/404: creat da/d14/d38/f78 x:0 0 0 2024-03-11T19:55:55.055 INFO:tasks.workunit.client.0.smithi107.stdout:8/319: dwrite dd/de/f15 [0,4194304] 0 2024-03-11T19:55:55.057 INFO:tasks.workunit.client.0.smithi107.stdout:8/320: rmdir dd/de/d1b/d20 39 2024-03-11T19:55:55.058 INFO:tasks.workunit.client.0.smithi107.stdout:5/405: getdents da/d14/d38/d59/d5f 0 2024-03-11T19:55:55.058 INFO:tasks.workunit.client.0.smithi107.stdout:5/406: chown da/d14 779166198 1 2024-03-11T19:55:55.058 INFO:tasks.workunit.client.0.smithi107.stdout:5/407: write da/d14/f68 [301918,60415] 0 2024-03-11T19:55:55.061 INFO:tasks.workunit.client.0.smithi107.stdout:0/312: dread de/d11/d23/d32/d34/d39/f53 [0,4194304] 0 2024-03-11T19:55:55.062 INFO:tasks.workunit.client.0.smithi107.stdout:8/321: creat dd/de/d49/f60 x:0 0 0 2024-03-11T19:55:55.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/322: creat dd/d13/d32/d51/f61 x:0 0 0 2024-03-11T19:55:55.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/323: mknod dd/de/d24/d28/d47/c62 0 2024-03-11T19:55:55.066 INFO:tasks.workunit.client.0.smithi107.stdout:8/324: read - dd/de/d49/f5b zero size 2024-03-11T19:55:55.070 INFO:tasks.workunit.client.0.smithi107.stdout:8/325: mknod dd/de/d1b/d2c/c63 0 2024-03-11T19:55:55.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/314: dread d4/d7/f17 [0,4194304] 0 2024-03-11T19:55:55.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/313: dread f5 [0,4194304] 0 2024-03-11T19:55:55.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/326: dread f4 [0,4194304] 0 2024-03-11T19:55:55.087 INFO:tasks.workunit.client.0.smithi107.stdout:8/327: readlink dd/d13/d32/d51/l5d 0 2024-03-11T19:55:55.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/328: truncate dd/d16/f33 106041 0 2024-03-11T19:55:55.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/329: truncate dd/de/d1b/d20/f29 2713362 0 2024-03-11T19:55:55.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/330: chown dd/de/d1b/d2c/l3e 296412745 1 2024-03-11T19:55:55.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/331: write dd/f58 [902644,112290] 0 2024-03-11T19:55:55.094 INFO:tasks.workunit.client.0.smithi107.stdout:8/332: symlink dd/d13/l64 0 2024-03-11T19:55:55.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/314: symlink de/d11/d23/d32/d34/d39/d55/l5c 0 2024-03-11T19:55:55.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/315: symlink de/d11/d23/d32/d34/d39/l5d 0 2024-03-11T19:55:55.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/300: dwrite d1/f2b [4194304,4194304] 0 2024-03-11T19:55:55.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/301: readlink d1/l17 0 2024-03-11T19:55:55.103 INFO:tasks.workunit.client.0.smithi107.stdout:6/299: dwrite d10/d16/d24/d2e/f46 [0,4194304] 0 2024-03-11T19:55:55.119 INFO:tasks.workunit.client.0.smithi107.stdout:1/354: dwrite d6/f10 [0,4194304] 0 2024-03-11T19:55:55.120 INFO:tasks.workunit.client.0.smithi107.stdout:4/315: dwrite d4/d7/d1f/d2b/d24/d26/d29/f32 [0,4194304] 0 2024-03-11T19:55:55.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/302: dread d1/d1d/f3e [0,4194304] 0 2024-03-11T19:55:55.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/303: fdatasync d1/f40 0 2024-03-11T19:55:55.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/304: write d1/d1a/d34/f42 [781338,65854] 0 2024-03-11T19:55:55.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/316: getdents d4/d7/d1f/d2b/d19 0 2024-03-11T19:55:55.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/317: creat d4/d7/d1f/d2b/f5d x:0 0 0 2024-03-11T19:55:55.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/333: dwrite dd/d13/f34 [0,4194304] 0 2024-03-11T19:55:55.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/355: dread d6/d19/d1e/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:55.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/356: chown d6/d28/f41 0 1 2024-03-11T19:55:55.143 INFO:tasks.workunit.client.0.smithi107.stdout:2/305: dread d1/d1a/d34/f42 [0,4194304] 0 2024-03-11T19:55:55.146 INFO:tasks.workunit.client.0.smithi107.stdout:6/300: rename f0 to d10/d19/f61 0 2024-03-11T19:55:55.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/301: creat d10/d3d/f62 x:0 0 0 2024-03-11T19:55:55.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/302: dread - d10/d16/d24/f58 zero size 2024-03-11T19:55:55.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/303: symlink d10/d16/d2b/d39/l63 0 2024-03-11T19:55:55.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/304: read - d10/d19/f5d zero size 2024-03-11T19:55:55.154 INFO:tasks.workunit.client.0.smithi107.stdout:0/316: dwrite de/d11/d23/d24/f2d [0,4194304] 0 2024-03-11T19:55:55.159 INFO:tasks.workunit.client.0.smithi107.stdout:4/318: dread d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:55.162 INFO:tasks.workunit.client.0.smithi107.stdout:4/319: mkdir d4/d7/db/d5e 0 2024-03-11T19:55:55.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/357: dread d6/f1b [0,4194304] 0 2024-03-11T19:55:55.163 INFO:tasks.workunit.client.0.smithi107.stdout:4/320: unlink d4/d7/d1f/d2b/l16 0 2024-03-11T19:55:55.167 INFO:tasks.workunit.client.0.smithi107.stdout:0/317: creat de/d11/d23/d32/d34/d39/d3a/f5e x:0 0 0 2024-03-11T19:55:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/321: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/322: chown d4/d7/d1f/d2b/d19/l1c 0 1 2024-03-11T19:55:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/323: truncate d4/d7/d1f/d2b/d19/f2a 1963949 0 2024-03-11T19:55:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/324: fsync d4/d7/d1f/d2b/d24/f3e 0 2024-03-11T19:55:55.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/306: dwrite d1/d1a/f2c [0,4194304] 0 2024-03-11T19:55:55.184 INFO:tasks.workunit.client.0.smithi107.stdout:2/307: rename d1/d1a/c1b to d1/d21/d35/c5f 0 2024-03-11T19:55:55.184 INFO:tasks.workunit.client.0.smithi107.stdout:2/308: truncate d1/d21/d35/f54 4601001 0 2024-03-11T19:55:55.193 INFO:tasks.workunit.client.0.smithi107.stdout:0/318: dread de/f3d [0,4194304] 0 2024-03-11T19:55:55.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/319: symlink de/l5f 0 2024-03-11T19:55:55.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/320: chown de/d11/d23/d24/l31 44 1 2024-03-11T19:55:55.198 INFO:tasks.workunit.client.0.smithi107.stdout:0/321: chown de/d11/c22 26920492 1 2024-03-11T19:55:55.198 INFO:tasks.workunit.client.0.smithi107.stdout:0/322: write de/f13 [997117,32784] 0 2024-03-11T19:55:55.199 INFO:tasks.workunit.client.0.smithi107.stdout:1/358: dwrite d6/f1d [0,4194304] 0 2024-03-11T19:55:55.200 INFO:tasks.workunit.client.0.smithi107.stdout:1/359: symlink d6/d19/l6c 0 2024-03-11T19:55:55.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/360: symlink d6/d5b/l6d 0 2024-03-11T19:55:55.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/361: readlink d6/d19/d1e/d27/l5d 0 2024-03-11T19:55:55.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/362: rename d6/l1a to d6/d19/d1e/d27/d2c/d32/l6e 0 2024-03-11T19:55:55.218 INFO:tasks.workunit.client.0.smithi107.stdout:4/325: dwrite d4/d7/d1f/d2b/f56 [0,4194304] 0 2024-03-11T19:55:55.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/326: mknod d4/d7/db/c5f 0 2024-03-11T19:55:55.223 INFO:tasks.workunit.client.0.smithi107.stdout:1/363: dread d6/d19/d1e/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:55.224 INFO:tasks.workunit.client.0.smithi107.stdout:2/309: dwrite d1/d1a/f2f [0,4194304] 0 2024-03-11T19:55:55.227 INFO:tasks.workunit.client.0.smithi107.stdout:2/310: creat d1/d1a/f60 x:0 0 0 2024-03-11T19:55:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:2/311: mknod d1/d1a/d1e/c61 0 2024-03-11T19:55:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:2/312: chown d1/d1a/d1e/d26/f30 7753504 1 2024-03-11T19:55:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:2/313: chown d1/d1d/f24 22721 1 2024-03-11T19:55:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:2/314: creat d1/d1a/d1e/d26/f62 x:0 0 0 2024-03-11T19:55:55.231 INFO:tasks.workunit.client.0.smithi107.stdout:2/315: mkdir d1/d1a/d34/d4f/d63 0 2024-03-11T19:55:55.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/316: rmdir d1/d1a 39 2024-03-11T19:55:55.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/317: mkdir d1/d1d/d64 0 2024-03-11T19:55:55.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/318: creat d1/d1a/f65 x:0 0 0 2024-03-11T19:55:55.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/364: write d6/f15 [111062,129756] 0 2024-03-11T19:55:55.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/319: unlink d1/c4d 0 2024-03-11T19:55:55.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/320: creat d1/d1a/d34/d4f/d63/f66 x:0 0 0 2024-03-11T19:55:55.240 INFO:tasks.workunit.client.0.smithi107.stdout:1/365: rename d6/d19/d1e/f4d to d6/d19/d1e/d27/d2c/d32/f6f 0 2024-03-11T19:55:55.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/366: truncate d6/d28/f41 2685163 0 2024-03-11T19:55:55.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/367: fsync d6/d19/d1e/f4e 0 2024-03-11T19:55:55.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/368: rename d6/d28/l2f to d6/d19/d1e/d27/d2c/d32/l70 0 2024-03-11T19:55:55.244 INFO:tasks.workunit.client.0.smithi107.stdout:1/369: symlink d6/d19/d1e/d27/d2c/l71 0 2024-03-11T19:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout:2/321: dread d1/d21/d35/f54 [0,4194304] 0 2024-03-11T19:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout:2/322: mkdir d1/d21/d35/d67 0 2024-03-11T19:55:55.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/370: dwrite d6/d19/d3d/d62/f65 [0,4194304] 0 2024-03-11T19:55:55.309 INFO:tasks.workunit.client.0.smithi107.stdout:1/371: mkdir d6/d19/d1e/d27/d2c/d72 0 2024-03-11T19:55:55.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/372: link d6/d28/f41 d6/d19/f73 0 2024-03-11T19:55:55.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/373: symlink d6/d19/d1e/d27/d2c/d32/d58/l74 0 2024-03-11T19:55:55.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/374: dread - d6/f54 zero size 2024-03-11T19:55:55.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/375: stat d6/d19/d1e/d27/d2c/d32/d58/l5a 0 2024-03-11T19:55:55.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/376: creat d6/d19/d1e/d27/d2c/d72/f75 x:0 0 0 2024-03-11T19:55:55.315 INFO:tasks.workunit.client.0.smithi107.stdout:4/327: dwrite d4/d7/f1d [0,4194304] 0 2024-03-11T19:55:55.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/377: rename d6/d19/c22 to d6/d19/d1e/c76 0 2024-03-11T19:55:55.317 INFO:tasks.workunit.client.0.smithi107.stdout:4/328: symlink d4/d7/db/d5e/l60 0 2024-03-11T19:55:55.320 INFO:tasks.workunit.client.0.smithi107.stdout:1/378: symlink d6/d19/d1e/d27/d2c/d72/l77 0 2024-03-11T19:55:55.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/379: creat d6/d5b/f78 x:0 0 0 2024-03-11T19:55:55.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/323: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:55:55.329 INFO:tasks.workunit.client.0.smithi107.stdout:2/324: write d1/d1a/d34/f42 [384666,77068] 0 2024-03-11T19:55:55.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/380: dread d6/d19/d1e/d27/d2c/d32/f6f [0,4194304] 0 2024-03-11T19:55:55.360 INFO:tasks.workunit.client.0.smithi107.stdout:4/329: dwrite d4/d7/d1f/d2b/d24/d26/f43 [4194304,4194304] 0 2024-03-11T19:55:55.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/330: symlink d4/d7/d2f/d33/d3a/l61 0 2024-03-11T19:55:55.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/331: chown d4/d7/d1f/d2b/d24/f3e 1857 1 2024-03-11T19:55:55.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/332: symlink d4/d7/d1f/d2b/d24/d26/d29/l62 0 2024-03-11T19:55:55.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/333: dread d4/d7/db/f21 [0,4194304] 0 2024-03-11T19:55:55.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/381: dwrite d6/f48 [0,4194304] 0 2024-03-11T19:55:55.400 INFO:tasks.workunit.client.0.smithi107.stdout:1/382: chown d6/d19/d39/l5f 964 1 2024-03-11T19:55:55.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/383: dread d6/d19/d1e/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:55.429 INFO:tasks.workunit.client.0.smithi107.stdout:4/334: write d4/f6 [574194,75225] 0 2024-03-11T19:55:55.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/335: truncate d4/f6 2464873 0 2024-03-11T19:55:55.437 INFO:tasks.workunit.client.0.smithi107.stdout:4/336: rmdir d4/d7/d1f/d2b/d24/d26/d29 39 2024-03-11T19:55:55.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/337: rename d4/d7/d1f/d2b/d24/d26/d29/l62 to d4/d7/db/d5e/l63 0 2024-03-11T19:55:55.474 INFO:tasks.workunit.client.0.smithi107.stdout:1/384: dwrite d6/d19/d1e/d27/d2c/f30 [0,4194304] 0 2024-03-11T19:55:55.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/338: dwrite d4/d7/d2f/d3b/f4e [0,4194304] 0 2024-03-11T19:55:55.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/385: dread d6/d19/d1e/d27/f38 [0,4194304] 0 2024-03-11T19:55:55.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/386: chown d6/d19/d1e/l31 2277651 1 2024-03-11T19:55:55.496 INFO:tasks.workunit.client.0.smithi107.stdout:4/339: creat d4/d7/d1f/d2b/d24/d26/d29/f64 x:0 0 0 2024-03-11T19:55:55.497 INFO:tasks.workunit.client.0.smithi107.stdout:4/340: getdents d4/d7/d2f/d33 0 2024-03-11T19:55:55.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/341: chown d4/d7/d2f/d3b/f58 6856473 1 2024-03-11T19:55:55.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/328: sync 2024-03-11T19:55:55.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/329: read d4/d1d/d1f/f2f [787008,92711] 0 2024-03-11T19:55:55.517 INFO:tasks.workunit.client.0.smithi107.stdout:9/330: symlink d4/d9/l80 0 2024-03-11T19:55:55.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/331: creat d4/d1d/d4b/f81 x:0 0 0 2024-03-11T19:55:55.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/332: creat d4/d1d/d1f/d28/d2c/f82 x:0 0 0 2024-03-11T19:55:55.532 INFO:tasks.workunit.client.0.smithi107.stdout:1/387: dread d6/d19/d1e/f29 [0,4194304] 0 2024-03-11T19:55:55.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/388: rename d6/d19/d1e/d27/d2c/f34 to d6/d19/d1e/d27/d2c/f79 0 2024-03-11T19:55:55.538 INFO:tasks.workunit.client.0.smithi107.stdout:1/389: rmdir d6/d19/d1e/d27 39 2024-03-11T19:55:55.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/390: creat d6/d19/d1e/d27/f7a x:0 0 0 2024-03-11T19:55:55.540 INFO:tasks.workunit.client.0.smithi107.stdout:1/391: creat d6/d19/d1e/d27/f7b x:0 0 0 2024-03-11T19:55:55.540 INFO:tasks.workunit.client.0.smithi107.stdout:1/392: chown d6/d19/c5e 100 1 2024-03-11T19:55:55.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/342: dwrite f3 [0,4194304] 0 2024-03-11T19:55:55.561 INFO:tasks.workunit.client.0.smithi107.stdout:1/393: dread d6/d19/d1e/f24 [0,4194304] 0 2024-03-11T19:55:55.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/394: creat d6/d28/f7c x:0 0 0 2024-03-11T19:55:55.565 INFO:tasks.workunit.client.0.smithi107.stdout:1/395: getdents d6/d19/d39 0 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:6/305: sync 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/373: sync 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/408: sync 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:0/323: sync 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:2/325: sync 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/374: stat d6/da/d16/f2c 0 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/409: truncate da/d14/d43/f6c 978192 0 2024-03-11T19:55:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/375: write d6/d9/f6d [526875,4872] 0 2024-03-11T19:55:55.567 INFO:tasks.workunit.client.0.smithi107.stdout:0/324: write de/f59 [291661,38508] 0 2024-03-11T19:55:55.567 INFO:tasks.workunit.client.0.smithi107.stdout:7/376: readlink d6/da/d16/d5c/l63 0 2024-03-11T19:55:55.567 INFO:tasks.workunit.client.0.smithi107.stdout:3/321: sync 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/334: sync 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/335: chown dd/l12 2 1 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/396: mkdir d6/d19/d1e/d27/d2c/d7d 0 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:7/377: write d6/d9/d4e/f59 [72768,115689] 0 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:6/306: creat d10/d19/d3b/f64 x:0 0 0 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:6/307: creat d10/d19/f65 x:0 0 0 2024-03-11T19:55:55.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/397: mkdir d6/d7e 0 2024-03-11T19:55:55.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/410: stat da/c33 0 2024-03-11T19:55:55.569 INFO:tasks.workunit.client.0.smithi107.stdout:8/336: unlink dd/de/f15 0 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/322: mknod d1/d19/d2d/c55 0 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/398: creat d6/d19/d1e/d27/d2c/d32/f7f x:0 0 0 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/399: chown d6/d19/d1e/d27/d2c/d32/l3a 215 1 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/400: read - d6/d28/f7c zero size 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/378: creat d6/d9/d13/d60/f72 x:0 0 0 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/379: chown d6 10242 1 2024-03-11T19:55:55.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/380: stat d6/da/d16/f3f 0 2024-03-11T19:55:55.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/401: creat d6/d28/f80 x:0 0 0 2024-03-11T19:55:55.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/381: creat d6/da/d16/d5c/f73 x:0 0 0 2024-03-11T19:55:55.581 INFO:tasks.workunit.client.0.smithi107.stdout:7/382: link d6/da/d16/d5c/f70 d6/d9/d57/d65/f74 0 2024-03-11T19:55:55.581 INFO:tasks.workunit.client.0.smithi107.stdout:8/337: creat dd/de/d1b/d20/d5e/f65 x:0 0 0 2024-03-11T19:55:55.582 INFO:tasks.workunit.client.0.smithi107.stdout:9/333: write d4/d1d/d1f/d28/f42 [1669686,111446] 0 2024-03-11T19:55:55.596 INFO:tasks.workunit.client.0.smithi107.stdout:9/334: mknod d4/d1d/d1f/d28/d2c/d56/c83 0 2024-03-11T19:55:55.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/343: write d4/d7/db/f49 [946112,94800] 0 2024-03-11T19:55:55.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/335: creat d4/d1d/d5c/d64/f84 x:0 0 0 2024-03-11T19:55:55.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/336: fdatasync d4/f16 0 2024-03-11T19:55:55.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/326: write d1/f2b [6632071,84488] 0 2024-03-11T19:55:55.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/327: chown d1/d1a/f2f 251706489 1 2024-03-11T19:55:55.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/328: chown d1/d38/l4b 337 1 2024-03-11T19:55:55.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/329: chown d1/d1a/d1e/d26/l3d 2485 1 2024-03-11T19:55:55.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/325: write de/d11/f4c [460564,110804] 0 2024-03-11T19:55:55.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/326: read - de/d11/f45 zero size 2024-03-11T19:55:55.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/337: link d4/d9/l4f d4/d1d/d1f/d28/d2c/d56/d6d/l85 0 2024-03-11T19:55:55.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/344: getdents d4/d7/d1f/d2b 0 2024-03-11T19:55:55.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/345: readlink d4/d7/db/lf 0 2024-03-11T19:55:55.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/346: dread - d4/d7/d1f/d2b/f5d zero size 2024-03-11T19:55:55.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/347: fdatasync d4/d7/d1f/d2b/d24/d26/d29/f64 0 2024-03-11T19:55:55.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/338: creat d4/d9/f86 x:0 0 0 2024-03-11T19:55:55.619 INFO:tasks.workunit.client.0.smithi107.stdout:8/338: rename dd/de/d24/c59 to dd/d13/d32/d51/c66 0 2024-03-11T19:55:55.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/330: rename d1/f6 to d1/d1a/d1e/d26/f68 0 2024-03-11T19:55:55.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/331: write d1/f15 [380060,9047] 0 2024-03-11T19:55:55.623 INFO:tasks.workunit.client.0.smithi107.stdout:2/332: truncate d1/d1a/d1e/d26/f32 7098852 0 2024-03-11T19:55:55.623 INFO:tasks.workunit.client.0.smithi107.stdout:8/339: symlink dd/de/d24/d28/d47/l67 0 2024-03-11T19:55:55.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/411: write da/d14/d38/d59/f69 [3954896,61441] 0 2024-03-11T19:55:55.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/412: chown da/d14/d38/d59 151788 1 2024-03-11T19:55:55.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/413: truncate da/d14/d38/d59/f64 915099 0 2024-03-11T19:55:55.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/414: creat da/d14/f79 x:0 0 0 2024-03-11T19:55:55.624 INFO:tasks.workunit.client.0.smithi107.stdout:7/383: read d6/f2f [2731594,118893] 0 2024-03-11T19:55:55.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/340: creat dd/de/d1b/d20/f68 x:0 0 0 2024-03-11T19:55:55.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/384: mknod d6/d21/c75 0 2024-03-11T19:55:55.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/385: truncate d6/d9/d13/f5e 619756 0 2024-03-11T19:55:55.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/323: dwrite d1/d10/d13/d4a/d4c/d1a/f20 [0,4194304] 0 2024-03-11T19:55:55.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/324: creat d1/d10/d21/d43/d48/f56 x:0 0 0 2024-03-11T19:55:55.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/308: dwrite d10/d16/f1e [4194304,4194304] 0 2024-03-11T19:55:55.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/325: creat d1/d10/f57 x:0 0 0 2024-03-11T19:55:55.646 INFO:tasks.workunit.client.0.smithi107.stdout:9/339: dread d4/d1d/d1f/d78/f1a [0,4194304] 0 2024-03-11T19:55:55.646 INFO:tasks.workunit.client.0.smithi107.stdout:9/340: fsync d4/fe 0 2024-03-11T19:55:55.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/333: truncate d1/d1a/d1e/d26/f32 1275558 0 2024-03-11T19:55:55.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/386: write d6/d9/d13/d1b/f3c [3135008,78428] 0 2024-03-11T19:55:55.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/327: dread de/d11/f1b [0,4194304] 0 2024-03-11T19:55:55.666 INFO:tasks.workunit.client.0.smithi107.stdout:7/387: mkdir d6/d9/d13/d1b/d76 0 2024-03-11T19:55:55.666 INFO:tasks.workunit.client.0.smithi107.stdout:5/415: dread da/d14/d38/d46/f61 [4194304,4194304] 0 2024-03-11T19:55:55.666 INFO:tasks.workunit.client.0.smithi107.stdout:7/388: mkdir d6/da/d4d/d77 0 2024-03-11T19:55:55.669 INFO:tasks.workunit.client.0.smithi107.stdout:5/416: symlink da/d14/d38/d46/l7a 0 2024-03-11T19:55:55.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/334: write d1/d21/f28 [1534509,68752] 0 2024-03-11T19:55:55.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/341: dread d4/d1d/d1f/d28/d2c/f34 [0,4194304] 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/342: mknod d4/d1c/d37/c87 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/343: read d4/d1d/d1f/d28/d2c/f46 [159362,43674] 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/344: write d4/d1d/d1f/d28/d2c/f5b [2837368,39218] 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/345: dread - d4/d1d/d1f/d28/f6e zero size 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/346: creat d4/d1d/d4b/f88 x:0 0 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/347: stat d4/d1d/d1f/f39 0 2024-03-11T19:55:55.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/348: fsync d4/d1d/d4b/f81 0 2024-03-11T19:55:55.684 INFO:tasks.workunit.client.0.smithi107.stdout:9/349: readlink d4/d1d/d1f/l69 0 2024-03-11T19:55:55.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/335: mknod d1/d1a/d34/d4f/c69 0 2024-03-11T19:55:55.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/328: dread de/d11/d23/d32/f54 [0,4194304] 0 2024-03-11T19:55:55.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/350: mknod d4/d1d/d1f/d78/c89 0 2024-03-11T19:55:55.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/336: mkdir d1/d1a/d34/d6a 0 2024-03-11T19:55:55.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/329: rmdir de/d11/d23/d24 39 2024-03-11T19:55:55.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/330: readlink l2 0 2024-03-11T19:55:55.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/351: mkdir d4/d1c/d47/d8a 0 2024-03-11T19:55:55.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/352: read - d4/d1d/d4b/f59 zero size 2024-03-11T19:55:55.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/353: creat d4/d1c/d37/f8b x:0 0 0 2024-03-11T19:55:55.688 INFO:tasks.workunit.client.0.smithi107.stdout:0/331: truncate de/f3d 3253642 0 2024-03-11T19:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout:9/354: link d4/d1d/d1f/l36 d4/d1d/d5c/l8c 0 2024-03-11T19:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout:9/355: chown d4/d1d/d1f/d28/d2c/d56/d6d/l6a 24910 1 2024-03-11T19:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/332: creat de/d11/d23/d24/f60 x:0 0 0 2024-03-11T19:55:55.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/356: symlink d4/d1c/d47/d8a/l8d 0 2024-03-11T19:55:55.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/357: readlink d4/d1c/d37/l44 0 2024-03-11T19:55:55.694 INFO:tasks.workunit.client.0.smithi107.stdout:9/358: rmdir d4/d1d/d1f/d78 39 2024-03-11T19:55:55.694 INFO:tasks.workunit.client.0.smithi107.stdout:9/359: readlink d4/d1d/d1f/d28/l66 0 2024-03-11T19:55:55.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/348: dwrite d4/d7/d2f/d3b/f58 [0,4194304] 0 2024-03-11T19:55:55.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/349: write d4/d7/d1f/d2b/d24/f5b [161548,14550] 0 2024-03-11T19:55:55.696 INFO:tasks.workunit.client.0.smithi107.stdout:7/389: write d6/ff [3972218,56608] 0 2024-03-11T19:55:55.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/360: creat d4/d9/d4d/f8e x:0 0 0 2024-03-11T19:55:55.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/361: truncate d4/d1d/d1f/d28/f6c 1014312 0 2024-03-11T19:55:55.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/333: creat de/d58/f61 x:0 0 0 2024-03-11T19:55:55.706 INFO:tasks.workunit.client.0.smithi107.stdout:0/334: symlink de/d11/d23/d32/d34/d39/d55/l62 0 2024-03-11T19:55:55.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/350: truncate d4/d7/d1f/d2b/d24/d26/d29/f32 2452426 0 2024-03-11T19:55:55.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/390: mkdir d6/d9/d13/d60/d61/d78 0 2024-03-11T19:55:55.709 INFO:tasks.workunit.client.0.smithi107.stdout:3/326: dwrite d1/d19/f44 [0,4194304] 0 2024-03-11T19:55:55.711 INFO:tasks.workunit.client.0.smithi107.stdout:0/335: link l2 de/d58/l63 0 2024-03-11T19:55:55.713 INFO:tasks.workunit.client.0.smithi107.stdout:0/336: fdatasync de/d11/d23/d32/d34/f36 0 2024-03-11T19:55:55.721 INFO:tasks.workunit.client.0.smithi107.stdout:3/327: creat d1/d10/d26/f58 x:0 0 0 2024-03-11T19:55:55.721 INFO:tasks.workunit.client.0.smithi107.stdout:3/328: fsync d1/d10/d26/f58 0 2024-03-11T19:55:55.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/337: rename de/c41 to de/d11/d23/d24/d4f/c64 0 2024-03-11T19:55:55.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/338: read de/f18 [52660,119746] 0 2024-03-11T19:55:55.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/417: dwrite da/d14/d18/d1e/f65 [0,4194304] 0 2024-03-11T19:55:55.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/339: link de/f17 de/d11/d23/d24/f65 0 2024-03-11T19:55:55.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/340: creat de/d11/d23/d32/d34/d39/d3a/f66 x:0 0 0 2024-03-11T19:55:55.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/341: fdatasync de/f13 0 2024-03-11T19:55:55.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/342: chown de/d11/d23/d32/d34/d39/f53 21607 1 2024-03-11T19:55:55.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/418: mknod da/d6e/c7b 0 2024-03-11T19:55:55.727 INFO:tasks.workunit.client.0.smithi107.stdout:5/419: mkdir da/d14/d77/d7c 0 2024-03-11T19:55:55.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/420: getdents da/d14/d38/d59 0 2024-03-11T19:55:55.734 INFO:tasks.workunit.client.0.smithi107.stdout:4/351: dread f3 [0,4194304] 0 2024-03-11T19:55:55.735 INFO:tasks.workunit.client.0.smithi107.stdout:4/352: creat d4/d7/d2f/f65 x:0 0 0 2024-03-11T19:55:55.738 INFO:tasks.workunit.client.0.smithi107.stdout:2/337: dwrite d1/d1a/f2f [4194304,4194304] 0 2024-03-11T19:55:55.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/353: creat d4/d7/d2f/d33/f66 x:0 0 0 2024-03-11T19:55:55.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/354: dread - d4/d7/d1f/f52 zero size 2024-03-11T19:55:55.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/391: dread d6/d21/f4c [0,4194304] 0 2024-03-11T19:55:55.741 INFO:tasks.workunit.client.0.smithi107.stdout:4/355: chown d4/d7/d2f/d33/d3a/l55 227 1 2024-03-11T19:55:55.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/392: symlink d6/da/d49/l79 0 2024-03-11T19:55:55.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/393: truncate d6/da/d16/f2c 1226383 0 2024-03-11T19:55:55.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/394: readlink d6/d9/l34 0 2024-03-11T19:55:55.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/395: write d6/da/f45 [141334,112418] 0 2024-03-11T19:55:55.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/396: truncate d6/da/d16/f36 4807466 0 2024-03-11T19:55:55.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/397: write d6/d9/f30 [587671,87281] 0 2024-03-11T19:55:55.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/398: write d6/da/d16/d5c/f68 [932111,108529] 0 2024-03-11T19:55:55.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/399: stat d6/d9/d57/d65/f6a 0 2024-03-11T19:55:55.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/400: chown d6/c66 50 1 2024-03-11T19:55:55.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/362: dwrite d4/f12 [0,4194304] 0 2024-03-11T19:55:55.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/363: creat d4/d1c/d37/f8f x:0 0 0 2024-03-11T19:55:55.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/364: rename d4/d9/f14 to d4/d1d/d1f/d78/f90 0 2024-03-11T19:55:55.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/365: creat d4/d1d/d1f/d28/f91 x:0 0 0 2024-03-11T19:55:55.764 INFO:tasks.workunit.client.0.smithi107.stdout:9/366: mknod d4/d1d/d5c/c92 0 2024-03-11T19:55:55.765 INFO:tasks.workunit.client.0.smithi107.stdout:9/367: mkdir d4/d1c/d47/d93 0 2024-03-11T19:55:55.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/368: unlink d4/d9/l4f 0 2024-03-11T19:55:55.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/369: write d4/d1d/d4b/f88 [611144,110118] 0 2024-03-11T19:55:55.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/343: dwrite de/d11/d23/d32/d34/d39/d3a/f57 [0,4194304] 0 2024-03-11T19:55:55.775 INFO:tasks.workunit.client.0.smithi107.stdout:5/421: dwrite da/d14/d38/d59/f1d [0,4194304] 0 2024-03-11T19:55:55.785 INFO:tasks.workunit.client.0.smithi107.stdout:3/329: dwrite d1/d19/d2d/f3c [0,4194304] 0 2024-03-11T19:55:55.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/401: dwrite d6/d9/d57/d65/f74 [0,4194304] 0 2024-03-11T19:55:55.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/422: link da/d14/d18/d1e/l5d da/d14/d38/d59/d54/d73/l7d 0 2024-03-11T19:55:55.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/423: link da/d14/l2e da/d14/d38/d59/d5f/l7e 0 2024-03-11T19:55:55.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/424: mkdir da/d14/d43/d7f 0 2024-03-11T19:55:55.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/425: creat da/d14/d38/d59/d5f/f80 x:0 0 0 2024-03-11T19:55:55.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/356: dread d4/d7/d2f/d3b/f58 [0,4194304] 0 2024-03-11T19:55:55.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/330: dread d1/d19/f32 [0,4194304] 0 2024-03-11T19:55:55.815 INFO:tasks.workunit.client.0.smithi107.stdout:2/338: dwrite d1/d1a/d1e/d26/f48 [0,4194304] 0 2024-03-11T19:55:55.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/331: creat d1/d10/d21/d43/d4f/f59 x:0 0 0 2024-03-11T19:55:55.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/339: symlink d1/d1a/d1e/l6b 0 2024-03-11T19:55:55.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/357: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:55.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/358: chown d4/d7/db/c12 60673151 1 2024-03-11T19:55:55.826 INFO:tasks.workunit.client.0.smithi107.stdout:9/370: dwrite d4/d1d/d1f/d28/d2c/f82 [0,4194304] 0 2024-03-11T19:55:55.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/344: dwrite de/f3d [0,4194304] 0 2024-03-11T19:55:55.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/345: truncate de/d11/d23/d32/d34/d39/d3a/f50 684294 0 2024-03-11T19:55:55.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/346: creat de/f67 x:0 0 0 2024-03-11T19:55:55.836 INFO:tasks.workunit.client.0.smithi107.stdout:9/371: creat d4/d1d/d1f/d28/d2c/d56/f94 x:0 0 0 2024-03-11T19:55:55.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/426: write da/d14/f3a [5285485,96251] 0 2024-03-11T19:55:55.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/402: read d6/da/d16/d5c/f54 [343567,96483] 0 2024-03-11T19:55:55.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/403: getdents d6/d9/d13/d1b/d76 0 2024-03-11T19:55:55.840 INFO:tasks.workunit.client.0.smithi107.stdout:7/404: mknod d6/da/d49/c7a 0 2024-03-11T19:55:55.840 INFO:tasks.workunit.client.0.smithi107.stdout:1/402: sync 2024-03-11T19:55:55.841 INFO:tasks.workunit.client.0.smithi107.stdout:6/309: sync 2024-03-11T19:55:55.842 INFO:tasks.workunit.client.0.smithi107.stdout:8/341: sync 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/405: creat d6/f7b x:0 0 0 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:2/340: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:2/341: chown d1/d1d/c1f 1400895 1 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/372: getdents d4/d1d/d1f/d28/d2c/d56 0 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:8/342: truncate dd/f2e 74501 0 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/406: creat d6/da/f7c x:0 0 0 2024-03-11T19:55:55.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/407: truncate d6/da/f7c 720565 0 2024-03-11T19:55:55.846 INFO:tasks.workunit.client.0.smithi107.stdout:2/342: creat d1/d1a/d1e/d26/f6c x:0 0 0 2024-03-11T19:55:55.846 INFO:tasks.workunit.client.0.smithi107.stdout:2/343: write d1/d1a/d1e/d26/f62 [649125,81687] 0 2024-03-11T19:55:55.846 INFO:tasks.workunit.client.0.smithi107.stdout:9/373: symlink d4/d1c/d37/l95 0 2024-03-11T19:55:55.848 INFO:tasks.workunit.client.0.smithi107.stdout:8/343: creat dd/f69 x:0 0 0 2024-03-11T19:55:55.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/403: rename d6/d19/d1e to d6/d7e/d81 0 2024-03-11T19:55:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:6/310: dread fd [0,4194304] 0 2024-03-11T19:55:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/344: rmdir dd/de/d24 39 2024-03-11T19:55:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/345: write dd/de/d1b/d20/f68 [933357,110217] 0 2024-03-11T19:55:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/346: creat dd/de/d1b/d20/f6a x:0 0 0 2024-03-11T19:55:55.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/404: rename d6/d19/d39 to d6/d19/d3d/d82 0 2024-03-11T19:55:55.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/405: write d6/d7e/d81/f5c [2856964,88938] 0 2024-03-11T19:55:55.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/406: dread - d6/f68 zero size 2024-03-11T19:55:55.859 INFO:tasks.workunit.client.0.smithi107.stdout:6/311: creat d10/d16/d2b/d39/f66 x:0 0 0 2024-03-11T19:55:55.872 INFO:tasks.workunit.client.0.smithi107.stdout:1/407: getdents d6 0 2024-03-11T19:55:55.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/312: mkdir d10/d3d/d54/d67 0 2024-03-11T19:55:55.886 INFO:tasks.workunit.client.0.smithi107.stdout:4/359: dwrite d4/d7/db/f21 [0,4194304] 0 2024-03-11T19:55:55.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/347: dwrite dd/f69 [0,4194304] 0 2024-03-11T19:55:55.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/348: chown dd/de/d49/d46 86809746 1 2024-03-11T19:55:55.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/408: dwrite d6/d9/d13/f67 [0,4194304] 0 2024-03-11T19:55:55.905 INFO:tasks.workunit.client.0.smithi107.stdout:0/347: dread de/d11/f4c [0,4194304] 0 2024-03-11T19:55:55.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/332: dwrite d1/d10/f57 [0,4194304] 0 2024-03-11T19:55:55.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/313: rename fd to d10/d16/d24/f68 0 2024-03-11T19:55:55.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/314: dread - d10/d16/d24/f5b zero size 2024-03-11T19:55:55.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/315: dread - d10/d19/d3b/f5e zero size 2024-03-11T19:55:55.907 INFO:tasks.workunit.client.0.smithi107.stdout:7/409: stat d6/d9/d13/c52 0 2024-03-11T19:55:55.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/344: dwrite d1/d1a/d1e/d26/f47 [0,4194304] 0 2024-03-11T19:55:55.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/316: unlink d10/d3d/f5a 0 2024-03-11T19:55:55.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/317: truncate d10/d19/d3b/f64 588535 0 2024-03-11T19:55:55.917 INFO:tasks.workunit.client.0.smithi107.stdout:2/345: unlink d1/l9 0 2024-03-11T19:55:55.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/374: dwrite d4/d1d/d1f/d78/f40 [0,4194304] 0 2024-03-11T19:55:55.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/375: write d4/d1d/d1f/d28/d2c/d56/f77 [4629744,3850] 0 2024-03-11T19:55:55.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/346: mknod d1/d1a/d1e/c6d 0 2024-03-11T19:55:55.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/347: chown d1/d1a/d1e/d41/l57 241 1 2024-03-11T19:55:55.926 INFO:tasks.workunit.client.0.smithi107.stdout:1/408: dwrite d6/f54 [0,4194304] 0 2024-03-11T19:55:55.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/348: truncate de/d11/d23/d24/f65 1898145 0 2024-03-11T19:55:55.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/349: dread - de/f44 zero size 2024-03-11T19:55:55.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/350: symlink de/l68 0 2024-03-11T19:55:55.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/351: creat de/d11/d23/d24/f69 x:0 0 0 2024-03-11T19:55:55.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/352: creat de/f6a x:0 0 0 2024-03-11T19:55:55.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/353: mknod de/d11/d23/d32/d34/d39/d3a/c6b 0 2024-03-11T19:55:55.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/360: rename d4/d7/d2f/d33/d3a/f44 to d4/d7/d1f/d2b/f67 0 2024-03-11T19:55:55.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/361: creat d4/d7/d2f/f68 x:0 0 0 2024-03-11T19:55:55.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/333: read d1/d10/d13/f3b [3197963,34096] 0 2024-03-11T19:55:55.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/354: rmdir de/d11/d23 39 2024-03-11T19:55:55.944 INFO:tasks.workunit.client.0.smithi107.stdout:7/410: rename d6/d9/d13 to d6/da/d16/d5c/d7d 0 2024-03-11T19:55:55.944 INFO:tasks.workunit.client.0.smithi107.stdout:7/411: fdatasync d6/f17 0 2024-03-11T19:55:55.944 INFO:tasks.workunit.client.0.smithi107.stdout:7/412: write d6/f5f [202863,83564] 0 2024-03-11T19:55:55.948 INFO:tasks.workunit.client.0.smithi107.stdout:7/413: link d6/d9/f6d d6/f7e 0 2024-03-11T19:55:55.948 INFO:tasks.workunit.client.0.smithi107.stdout:0/355: creat de/d11/d23/d32/d34/f6c x:0 0 0 2024-03-11T19:55:55.949 INFO:tasks.workunit.client.0.smithi107.stdout:7/414: mknod d6/d9/d57/c7f 0 2024-03-11T19:55:55.949 INFO:tasks.workunit.client.0.smithi107.stdout:7/415: fdatasync d6/d21/f3a 0 2024-03-11T19:55:55.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/318: dread d10/d19/f61 [0,4194304] 0 2024-03-11T19:55:55.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/319: creat d10/d19/d38/d60/f69 x:0 0 0 2024-03-11T19:55:55.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/320: chown d10/d16/d24/d2e/f46 510 1 2024-03-11T19:55:55.959 INFO:tasks.workunit.client.0.smithi107.stdout:0/356: link de/l68 de/d11/d23/d24/d4f/l6d 0 2024-03-11T19:55:55.960 INFO:tasks.workunit.client.0.smithi107.stdout:9/376: dread d4/d1d/d1f/d28/d2c/d56/d6d/f2d [0,4194304] 0 2024-03-11T19:55:55.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/321: mkdir d10/d16/d2b/d39/d6a 0 2024-03-11T19:55:55.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/322: chown d10/d16/d2b/d39/d6a 1310727965 1 2024-03-11T19:55:55.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/416: rmdir d6/da/d4d 39 2024-03-11T19:55:55.962 INFO:tasks.workunit.client.0.smithi107.stdout:8/349: dwrite f4 [4194304,4194304] 0 2024-03-11T19:55:55.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/417: write d6/d21/f4c [1457453,32484] 0 2024-03-11T19:55:55.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/357: truncate de/f40 431124 0 2024-03-11T19:55:55.964 INFO:tasks.workunit.client.0.smithi107.stdout:7/418: mkdir d6/da/d16/d5c/d7d/d1b/d76/d80 0 2024-03-11T19:55:55.966 INFO:tasks.workunit.client.0.smithi107.stdout:0/358: creat de/d11/d23/d32/d34/d39/f6e x:0 0 0 2024-03-11T19:55:55.966 INFO:tasks.workunit.client.0.smithi107.stdout:4/362: dread d4/d7/d2f/d3b/f4e [0,4194304] 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:0/359: mknod de/d11/d23/d32/d34/c6f 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:0/360: write de/d11/f45 [935100,44307] 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/363: symlink d4/d7/d2f/d3c/l69 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/364: creat d4/d7/d1f/d2b/d24/f6a x:0 0 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:6/323: symlink d10/d19/d38/d40/l6b 0 2024-03-11T19:55:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/350: mknod dd/de/d49/c6b 0 2024-03-11T19:55:55.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/324: creat d10/d16/d1f/d51/f6c x:0 0 0 2024-03-11T19:55:55.980 INFO:tasks.workunit.client.0.smithi107.stdout:9/377: rename d4/d1d/d79 to d4/d1d/d1f/d28/d2c/d96 0 2024-03-11T19:55:55.984 INFO:tasks.workunit.client.0.smithi107.stdout:9/378: mkdir d4/d1d/d5c/d97 0 2024-03-11T19:55:55.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/361: dread de/f13 [0,4194304] 0 2024-03-11T19:55:55.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/362: stat de/d11/d23/d26/c3e 0 2024-03-11T19:55:55.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/363: mknod de/d11/d23/d32/d34/d39/d55/c70 0 2024-03-11T19:55:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/364: chown de/d11/d23/d32/d34/f36 715 1 2024-03-11T19:55:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/365: chown de/c15 433 1 2024-03-11T19:55:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/409: dread d6/d7e/d81/d27/f3e [0,4194304] 0 2024-03-11T19:55:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/410: chown d6/d7e/d81/d27/d2c/d72/l77 1340 1 2024-03-11T19:55:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/411: truncate d6/d7e/d81/f4e 995050 0 2024-03-11T19:55:55.994 INFO:tasks.workunit.client.0.smithi107.stdout:9/379: dread d4/d1d/d1f/d28/f6c [0,4194304] 0 2024-03-11T19:55:55.994 INFO:tasks.workunit.client.0.smithi107.stdout:9/380: truncate d4/d1d/d4b/f88 146906 0 2024-03-11T19:55:55.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/348: dwrite d1/f15 [0,4194304] 0 2024-03-11T19:55:55.995 INFO:tasks.workunit.client.0.smithi107.stdout:0/366: getdents de 0 2024-03-11T19:55:55.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/349: creat d1/d21/d35/d67/f6e x:0 0 0 2024-03-11T19:55:55.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/350: write d1/d1d/f5b [1014794,117362] 0 2024-03-11T19:55:55.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/351: write d1/d21/d35/d67/f6e [595079,108242] 0 2024-03-11T19:55:55.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/367: rename de/d11/f4d to de/d11/d23/f71 0 2024-03-11T19:55:56.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/368: mkdir de/d11/d23/d32/d34/d39/d3a/d72 0 2024-03-11T19:55:56.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/334: dwrite d1/d10/f57 [0,4194304] 0 2024-03-11T19:55:56.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/369: rename de/d11/d23/d32/d34/d39/d55/l62 to de/d11/d23/d32/d34/d39/d3a/l73 0 2024-03-11T19:55:56.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/335: unlink d1/d19/d2d/f3c 0 2024-03-11T19:55:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:0/370: mkdir de/d11/d74 0 2024-03-11T19:55:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:0/371: chown de/d11/d23/d32/d34/d39/l52 1720 1 2024-03-11T19:55:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/336: symlink d1/l5a 0 2024-03-11T19:55:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/337: fsync d1/fa 0 2024-03-11T19:55:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/338: write d1/d10/d2a/f38 [937976,17272] 0 2024-03-11T19:55:56.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:55 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:56.011 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:55 smithi122 ceph-mon[36397]: pgmap v78: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 343 op/s 2024-03-11T19:55:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:3/339: rename d1/fa to d1/d10/d26/d17/f5b 0 2024-03-11T19:55:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:3/340: readlink d1/d10/d26/l4b 0 2024-03-11T19:55:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:3/341: stat d1/d19/c3f 0 2024-03-11T19:55:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/352: dread d1/d21/f2e [0,4194304] 0 2024-03-11T19:55:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/353: creat d1/d1a/d34/d4f/d63/f6f x:0 0 0 2024-03-11T19:55:56.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/342: chown d1/d10/d4e 31 1 2024-03-11T19:55:56.016 INFO:tasks.workunit.client.0.smithi107.stdout:2/354: symlink d1/d1a/d1e/d41/l70 0 2024-03-11T19:55:56.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/355: mknod d1/d38/c71 0 2024-03-11T19:55:56.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/356: readlink d1/d1a/d1e/d26/l3d 0 2024-03-11T19:55:56.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/357: read d1/f40 [389156,32077] 0 2024-03-11T19:55:56.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/358: symlink d1/d1d/l72 0 2024-03-11T19:55:56.034 INFO:tasks.workunit.client.0.smithi107.stdout:3/343: write d1/d10/d13/d4a/d4c/d1a/f20 [982931,23586] 0 2024-03-11T19:55:56.034 INFO:tasks.workunit.client.0.smithi107.stdout:1/412: dread d6/f54 [0,4194304] 0 2024-03-11T19:55:56.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:55 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:56.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:55 smithi145 ceph-mon[36730]: pgmap v78: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 343 op/s 2024-03-11T19:55:56.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/344: rename d1/d10/d13/f3b to d1/d10/d13/d4a/f5c 0 2024-03-11T19:55:56.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/413: rmdir d6/d7e/d81/d27/d2c/d67 0 2024-03-11T19:55:56.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/345: mkdir d1/d19/d5d 0 2024-03-11T19:55:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/346: getdents d1/d10/d21 0 2024-03-11T19:55:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:0/372: dread de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:56.039 INFO:tasks.workunit.client.0.smithi107.stdout:3/347: symlink d1/d10/d21/l5e 0 2024-03-11T19:55:56.039 INFO:tasks.workunit.client.0.smithi107.stdout:3/348: symlink d1/d10/l5f 0 2024-03-11T19:55:56.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:55 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:55:56.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:55 smithi107 ceph-mon[31712]: pgmap v78: 129 pgs: 129 active+clean; 463 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 55 MiB/s wr, 343 op/s 2024-03-11T19:55:56.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/365: dwrite d4/d7/d1f/d2b/d24/d26/d29/f30 [0,4194304] 0 2024-03-11T19:55:56.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/366: symlink d4/d7/d1f/d2b/d24/l6b 0 2024-03-11T19:55:56.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/367: write d4/d7/d2f/d33/f66 [971180,67494] 0 2024-03-11T19:55:56.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/349: dread d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:55:56.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/368: write d4/d7/d1f/d2b/d24/d26/f51 [6126511,74591] 0 2024-03-11T19:55:56.080 INFO:tasks.workunit.client.0.smithi107.stdout:4/369: truncate d4/d7/d1f/d2b/d24/d26/f51 3474749 0 2024-03-11T19:55:56.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/370: mknod d4/d7/d2f/d3b/c6c 0 2024-03-11T19:55:56.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/414: dwrite d6/d19/f43 [0,4194304] 0 2024-03-11T19:55:56.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/325: dwrite d10/d16/d24/f68 [0,4194304] 0 2024-03-11T19:55:56.117 INFO:tasks.workunit.client.0.smithi107.stdout:8/351: dwrite f4 [0,4194304] 0 2024-03-11T19:55:56.117 INFO:tasks.workunit.client.0.smithi107.stdout:8/352: write dd/d16/f19 [3574154,59548] 0 2024-03-11T19:55:56.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/353: unlink dd/d13/d32/f45 0 2024-03-11T19:55:56.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/373: dwrite de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:56.126 INFO:tasks.workunit.client.0.smithi107.stdout:0/374: write de/d11/f2c [4488249,108514] 0 2024-03-11T19:55:56.126 INFO:tasks.workunit.client.0.smithi107.stdout:0/375: fsync de/d11/d23/d26/f42 0 2024-03-11T19:55:56.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/415: dwrite d6/d7e/d81/d27/d2c/d72/f75 [0,4194304] 0 2024-03-11T19:55:56.147 INFO:tasks.workunit.client.0.smithi107.stdout:6/326: dread fa [0,4194304] 0 2024-03-11T19:55:56.150 INFO:tasks.workunit.client.0.smithi107.stdout:9/381: dwrite d4/d1d/d1f/d28/d2c/f5b [0,4194304] 0 2024-03-11T19:55:56.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/382: symlink d4/d1c/d47/d93/l98 0 2024-03-11T19:55:56.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/354: dwrite f4 [4194304,4194304] 0 2024-03-11T19:55:56.160 INFO:tasks.workunit.client.0.smithi107.stdout:8/355: dread - dd/de/f48 zero size 2024-03-11T19:55:56.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/356: dread - dd/de/f4e zero size 2024-03-11T19:55:56.165 INFO:tasks.workunit.client.0.smithi107.stdout:9/383: dread d4/d1d/d1f/d78/f74 [0,4194304] 0 2024-03-11T19:55:56.165 INFO:tasks.workunit.client.0.smithi107.stdout:9/384: dread - d4/d1d/d1f/d28/f91 zero size 2024-03-11T19:55:56.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/385: symlink d4/d1c/d47/d8a/l99 0 2024-03-11T19:55:56.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/386: symlink d4/d1c/d47/d8a/l9a 0 2024-03-11T19:55:56.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/387: symlink d4/d1c/d47/l9b 0 2024-03-11T19:55:56.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/388: write d4/d9/f86 [712595,18028] 0 2024-03-11T19:55:56.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/389: creat d4/d1c/d47/f9c x:0 0 0 2024-03-11T19:55:56.178 INFO:tasks.workunit.client.0.smithi107.stdout:1/416: dwrite d6/d19/f73 [0,4194304] 0 2024-03-11T19:55:56.179 INFO:tasks.workunit.client.0.smithi107.stdout:1/417: creat d6/d28/d60/f83 x:0 0 0 2024-03-11T19:55:56.181 INFO:tasks.workunit.client.0.smithi107.stdout:5/427: sync 2024-03-11T19:55:56.182 INFO:tasks.workunit.client.0.smithi107.stdout:5/428: chown da/d14/d38/d59/c3b 175763528 1 2024-03-11T19:55:56.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/429: symlink da/d14/d43/d7f/l81 0 2024-03-11T19:55:56.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/430: dread - da/d14/d38/d59/d54/f57 zero size 2024-03-11T19:55:56.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/431: stat da/d14/d18/d44/c50 0 2024-03-11T19:55:56.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/432: chown da/d14/d18/l74 25533 1 2024-03-11T19:55:56.196 INFO:tasks.workunit.client.0.smithi107.stdout:8/357: dwrite dd/de/d1b/f1d [0,4194304] 0 2024-03-11T19:55:56.204 INFO:tasks.workunit.client.0.smithi107.stdout:6/327: dwrite d10/d16/d24/f68 [0,4194304] 0 2024-03-11T19:55:56.204 INFO:tasks.workunit.client.0.smithi107.stdout:6/328: chown d10/d3d/d54/d67 233 1 2024-03-11T19:55:56.204 INFO:tasks.workunit.client.0.smithi107.stdout:8/358: dread dd/de/d24/d28/f4a [0,4194304] 0 2024-03-11T19:55:56.209 INFO:tasks.workunit.client.0.smithi107.stdout:9/390: dwrite d4/d1d/d1f/d28/f6e [0,4194304] 0 2024-03-11T19:55:56.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/391: rename d4/d1d/d1f/d78/f40 to d4/d1d/d5c/d64/f9d 0 2024-03-11T19:55:56.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/392: readlink d4/d1c/l71 0 2024-03-11T19:55:56.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/393: chown d4/d1c/d47/l9b 5 1 2024-03-11T19:55:56.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/394: dread - d4/d1d/d1f/f39 zero size 2024-03-11T19:55:56.213 INFO:tasks.workunit.client.0.smithi107.stdout:6/329: link d10/d19/d38/f5c d10/d16/d2b/d39/f6d 0 2024-03-11T19:55:56.214 INFO:tasks.workunit.client.0.smithi107.stdout:6/330: mkdir d10/d16/d2b/d39/d6a/d6e 0 2024-03-11T19:55:56.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/433: write da/f71 [1124476,59344] 0 2024-03-11T19:55:56.216 INFO:tasks.workunit.client.0.smithi107.stdout:6/331: symlink d10/d19/d3b/l6f 0 2024-03-11T19:55:56.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/434: write da/fb [72360,116013] 0 2024-03-11T19:55:56.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/332: dread d10/d16/d24/d2e/f36 [0,4194304] 0 2024-03-11T19:55:56.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/333: fsync d10/d16/f33 0 2024-03-11T19:55:56.246 INFO:tasks.workunit.client.0.smithi107.stdout:6/334: mkdir d10/d19/d3b/d70 0 2024-03-11T19:55:56.247 INFO:tasks.workunit.client.0.smithi107.stdout:6/335: creat d10/f71 x:0 0 0 2024-03-11T19:55:56.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/336: unlink d10/d16/d1f/l57 0 2024-03-11T19:55:56.251 INFO:tasks.workunit.client.0.smithi107.stdout:9/395: dwrite d4/d1d/d1f/d28/d2c/d96/f7a [0,4194304] 0 2024-03-11T19:55:56.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/396: unlink d4/d1d/d1f/d28/d2c/c3e 0 2024-03-11T19:55:56.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/397: chown d4/d1d/d1f/d28/l29 154041170 1 2024-03-11T19:55:56.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/337: rename d10/d19/d38/c48 to d10/d19/d3b/c72 0 2024-03-11T19:55:56.257 INFO:tasks.workunit.client.0.smithi107.stdout:6/338: creat d10/d19/d3b/f73 x:0 0 0 2024-03-11T19:55:56.257 INFO:tasks.workunit.client.0.smithi107.stdout:6/339: chown d10/d16/d24/d2e/f34 79531623 1 2024-03-11T19:55:56.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/435: dread da/fb [0,4194304] 0 2024-03-11T19:55:56.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/436: chown da 174808827 1 2024-03-11T19:55:56.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/437: chown da/d14/f3a 15 1 2024-03-11T19:55:56.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/438: rename da/f36 to da/d6e/f82 0 2024-03-11T19:55:56.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/439: creat da/d14/d18/d44/f83 x:0 0 0 2024-03-11T19:55:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/440: mknod da/d14/d18/d44/c84 0 2024-03-11T19:55:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/441: creat da/f85 x:0 0 0 2024-03-11T19:55:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/442: creat da/f86 x:0 0 0 2024-03-11T19:55:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/443: write da/d6e/f82 [9391900,58494] 0 2024-03-11T19:55:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/444: dread - da/d14/d38/d59/d5f/f80 zero size 2024-03-11T19:55:56.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/445: creat da/d14/d38/f87 x:0 0 0 2024-03-11T19:55:56.266 INFO:tasks.workunit.client.0.smithi107.stdout:5/446: rename da/d14/d18/d44/f83 to da/d14/d77/f88 0 2024-03-11T19:55:56.268 INFO:tasks.workunit.client.0.smithi107.stdout:5/447: rename da/d14/l2e to da/d6e/l89 0 2024-03-11T19:55:56.268 INFO:tasks.workunit.client.0.smithi107.stdout:5/448: stat da/f32 0 2024-03-11T19:55:56.268 INFO:tasks.workunit.client.0.smithi107.stdout:5/449: read - da/d14/d18/f4c zero size 2024-03-11T19:55:56.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/398: dread f1 [0,4194304] 0 2024-03-11T19:55:56.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/450: dread da/d14/d18/f27 [0,4194304] 0 2024-03-11T19:55:56.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/419: sync 2024-03-11T19:55:56.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/451: dread da/d31/d42/f28 [0,4194304] 0 2024-03-11T19:55:56.316 INFO:tasks.workunit.client.0.smithi107.stdout:5/452: creat da/d14/d38/d59/d54/d73/f8a x:0 0 0 2024-03-11T19:55:56.317 INFO:tasks.workunit.client.0.smithi107.stdout:5/453: rmdir da/d14/d38/d46 39 2024-03-11T19:55:56.320 INFO:tasks.workunit.client.0.smithi107.stdout:5/454: getdents da/d14/d18/d1e 0 2024-03-11T19:55:56.320 INFO:tasks.workunit.client.0.smithi107.stdout:5/455: fdatasync da/d14/d18/f1c 0 2024-03-11T19:55:56.321 INFO:tasks.workunit.client.0.smithi107.stdout:5/456: mkdir da/d14/d18/d44/d8b 0 2024-03-11T19:55:56.322 INFO:tasks.workunit.client.0.smithi107.stdout:5/457: write da/d14/d43/f6c [1133208,63515] 0 2024-03-11T19:55:56.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/399: dwrite d4/d1d/d4b/f88 [0,4194304] 0 2024-03-11T19:55:56.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/400: truncate d4/fe 5185414 0 2024-03-11T19:55:56.354 INFO:tasks.workunit.client.0.smithi107.stdout:7/420: dwrite d6/da/d16/d5c/f54 [0,4194304] 0 2024-03-11T19:55:56.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/421: write d6/d9/f30 [496709,10128] 0 2024-03-11T19:55:56.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/422: rename d6/d9/d57 to d6/d9/d57/d65/d81 22 2024-03-11T19:55:56.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/423: truncate d6/d9/f4f 640053 0 2024-03-11T19:55:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/458: dwrite da/d31/f75 [0,4194304] 0 2024-03-11T19:55:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/459: readlink da/d14/d38/d46/l53 0 2024-03-11T19:55:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/460: dread - da/d14/d18/f76 zero size 2024-03-11T19:55:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/461: chown da/d14/l22 743 1 2024-03-11T19:55:56.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/424: getdents d6/d21 0 2024-03-11T19:55:56.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/425: truncate d6/f5f 1289989 0 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/426: rename d6/da/d16/d5c/d7d/d60/d61/c62 to d6/d21/c82 0 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/427: symlink d6/da/d16/d5c/d7d/d1b/d76/l83 0 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/371: sync 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:2/359: sync 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/350: sync 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:2/360: write d1/fc [4839796,81214] 0 2024-03-11T19:55:56.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/428: mknod d6/da/d16/d5c/c84 0 2024-03-11T19:55:56.379 INFO:tasks.workunit.client.0.smithi107.stdout:7/429: write d6/d9/d57/d65/f6a [253162,75612] 0 2024-03-11T19:55:56.379 INFO:tasks.workunit.client.0.smithi107.stdout:3/351: mknod d1/d10/d26/c60 0 2024-03-11T19:55:56.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/361: unlink d1/d1a/d1e/d26/f6c 0 2024-03-11T19:55:56.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/430: link d6/da/d16/d5c/d7d/f51 d6/d9/f85 0 2024-03-11T19:55:56.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/352: creat d1/d10/d13/f61 x:0 0 0 2024-03-11T19:55:56.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/431: creat d6/d21/f86 x:0 0 0 2024-03-11T19:55:56.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/432: chown d6/da/d16/f6f 451168 1 2024-03-11T19:55:56.384 INFO:tasks.workunit.client.0.smithi107.stdout:7/433: rename d6/da/d16/c18 to d6/d9/d57/c87 0 2024-03-11T19:55:56.385 INFO:tasks.workunit.client.0.smithi107.stdout:3/353: symlink d1/d10/l62 0 2024-03-11T19:55:56.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/434: truncate d6/da/d16/d5c/f70 1468495 0 2024-03-11T19:55:56.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/354: link d1/d10/d26/f25 d1/d45/f63 0 2024-03-11T19:55:56.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/355: creat d1/d19/d2d/d4d/f64 x:0 0 0 2024-03-11T19:55:56.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/435: mknod d6/da/d16/d5c/d7d/d1b/d76/c88 0 2024-03-11T19:55:56.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/436: chown d6/da/d16/d5c/d7d/d60 733124765 1 2024-03-11T19:55:56.392 INFO:tasks.workunit.client.0.smithi107.stdout:7/437: truncate d6/f7e 161082 0 2024-03-11T19:55:56.392 INFO:tasks.workunit.client.0.smithi107.stdout:4/372: read d4/d7/f8 [788322,28396] 0 2024-03-11T19:55:56.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/356: rename d1/d10/l15 to d1/d19/d5d/l65 0 2024-03-11T19:55:56.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/438: symlink d6/d9/l89 0 2024-03-11T19:55:56.406 INFO:tasks.workunit.client.0.smithi107.stdout:4/373: mkdir d4/d7/d2f/d33/d6d 0 2024-03-11T19:55:56.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/374: readlink d4/d7/db/l10 0 2024-03-11T19:55:56.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/439: link d6/d21/f4c d6/da/f8a 0 2024-03-11T19:55:56.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/440: chown d6/da/d16/d5c/d7d/d1b/c25 455 1 2024-03-11T19:55:56.408 INFO:tasks.workunit.client.0.smithi107.stdout:5/462: dwrite da/d14/d38/d59/d5f/f6a [0,4194304] 0 2024-03-11T19:55:56.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/375: mknod d4/d7/d1f/d2b/c6e 0 2024-03-11T19:55:56.410 INFO:tasks.workunit.client.0.smithi107.stdout:3/357: mknod d1/d19/d5d/c66 0 2024-03-11T19:55:56.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/463: mknod da/d14/d38/d59/d54/c8c 0 2024-03-11T19:55:56.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/464: readlink da/d14/d18/d1e/l2a 0 2024-03-11T19:55:56.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/401: dwrite d4/f54 [0,4194304] 0 2024-03-11T19:55:56.412 INFO:tasks.workunit.client.0.smithi107.stdout:7/441: symlink d6/da/d16/d5c/d7d/d1b/d76/d80/l8b 0 2024-03-11T19:55:56.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/376: mknod d4/d7/d1f/d2b/c6f 0 2024-03-11T19:55:56.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/377: write d4/d7/d1f/d2b/f5d [790955,115388] 0 2024-03-11T19:55:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/465: symlink da/d14/d18/d6d/l8d 0 2024-03-11T19:55:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/466: chown da/d14/d38/d46 13466 1 2024-03-11T19:55:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/402: write d4/d1d/d1f/d78/f1a [217071,62554] 0 2024-03-11T19:55:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/403: chown d4/d1d/d1f/d28/f6c 9 1 2024-03-11T19:55:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/404: write d4/d1d/d5c/d64/f84 [806293,20250] 0 2024-03-11T19:55:56.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/378: truncate d4/d7/f17 1694267 0 2024-03-11T19:55:56.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/379: read - d4/d7/d1f/d2b/d24/f6a zero size 2024-03-11T19:55:56.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/380: fdatasync d4/d7/d1f/d2b/d19/f23 0 2024-03-11T19:55:56.417 INFO:tasks.workunit.client.0.smithi107.stdout:9/405: read d4/d1d/f52 [378670,62143] 0 2024-03-11T19:55:56.418 INFO:tasks.workunit.client.0.smithi107.stdout:9/406: dread - d4/d1d/d1f/d28/f91 zero size 2024-03-11T19:55:56.418 INFO:tasks.workunit.client.0.smithi107.stdout:9/407: creat d4/d1c/f9e x:0 0 0 2024-03-11T19:55:56.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/358: rename d1/d10/c29 to d1/d10/d13/d4a/d4c/c67 0 2024-03-11T19:55:56.419 INFO:tasks.workunit.client.0.smithi107.stdout:5/467: link l7 da/d14/d43/d47/l8e 0 2024-03-11T19:55:56.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/408: rename d4/d1c/d47/d93 to d4/d9/d9f 0 2024-03-11T19:55:56.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/359: getdents d1/d10/d21 0 2024-03-11T19:55:56.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/409: write d4/d1c/d37/f8b [69685,101947] 0 2024-03-11T19:55:56.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/410: fsync d4/d1c/d37/f8f 0 2024-03-11T19:55:56.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/360: mknod d1/d10/c68 0 2024-03-11T19:55:56.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/411: symlink d4/d1d/d1f/d28/d2c/d96/la0 0 2024-03-11T19:55:56.434 INFO:tasks.workunit.client.0.smithi107.stdout:9/412: mknod d4/d1c/d47/d8a/ca1 0 2024-03-11T19:55:56.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/413: creat d4/d1d/d1f/d28/fa2 x:0 0 0 2024-03-11T19:55:56.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/414: rename d4/d1d/f52 to d4/d1d/d1f/d28/d2c/fa3 0 2024-03-11T19:55:56.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/415: read - d4/d1c/d37/f8f zero size 2024-03-11T19:55:56.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/416: chown d4/d1d/d1f/d28/l29 79818278 1 2024-03-11T19:55:56.440 INFO:tasks.workunit.client.0.smithi107.stdout:9/417: creat d4/d1d/d1f/d28/d2c/d96/fa4 x:0 0 0 2024-03-11T19:55:56.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/362: dwrite d1/d21/f28 [0,4194304] 0 2024-03-11T19:55:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/363: mknod d1/d1a/d34/d6a/c73 0 2024-03-11T19:55:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/364: creat d1/d1a/f74 x:0 0 0 2024-03-11T19:55:56.452 INFO:tasks.workunit.client.0.smithi107.stdout:1/418: sync 2024-03-11T19:55:56.454 INFO:tasks.workunit.client.0.smithi107.stdout:0/376: sync 2024-03-11T19:55:56.454 INFO:tasks.workunit.client.0.smithi107.stdout:4/381: dwrite d4/d7/d1f/d2b/d24/d26/d29/f64 [0,4194304] 0 2024-03-11T19:55:56.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/442: dwrite d6/f2f [4194304,4194304] 0 2024-03-11T19:55:56.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/443: creat d6/d9/f8c x:0 0 0 2024-03-11T19:55:56.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/444: read d6/ff [2246144,60655] 0 2024-03-11T19:55:56.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/445: write d6/f5f [2137492,80641] 0 2024-03-11T19:55:56.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/446: write d6/d9/f6d [856358,92774] 0 2024-03-11T19:55:56.456 INFO:tasks.workunit.client.0.smithi107.stdout:4/382: creat d4/d7/d1f/d2b/d19/f70 x:0 0 0 2024-03-11T19:55:56.458 INFO:tasks.workunit.client.0.smithi107.stdout:3/361: write d1/d10/d13/d4a/d4c/f2f [6873547,50059] 0 2024-03-11T19:55:56.458 INFO:tasks.workunit.client.0.smithi107.stdout:3/362: creat d1/d10/d21/d43/d48/f69 x:0 0 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/363: mknod d1/c6a 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/364: fsync d1/d10/d13/d4a/f5c 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/365: fdatasync d1/f5 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/366: stat d1/d10/d13/d4a/d4c/d49 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/367: dread - d1/d19/d2d/d4d/f64 zero size 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/368: fdatasync d1/d10/d13/d4a/d4c/d1a/f20 0 2024-03-11T19:55:56.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/369: fdatasync d1/d10/d13/d4a/d4c/d1a/f41 0 2024-03-11T19:55:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:3/370: unlink d1/d19/d2d/d4d/f64 0 2024-03-11T19:55:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:2/365: write d1/f49 [2875919,470] 0 2024-03-11T19:55:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/419: link d6/d7e/d81/l31 d6/d7e/d81/d27/d2c/d72/l84 0 2024-03-11T19:55:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:2/366: truncate d1/d1a/f74 873938 0 2024-03-11T19:55:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/420: creat d6/d5b/f85 x:0 0 0 2024-03-11T19:55:56.466 INFO:tasks.workunit.client.0.smithi107.stdout:3/371: mkdir d1/d10/d13/d4a/d4c/d1a/d6b 0 2024-03-11T19:55:56.468 INFO:tasks.workunit.client.0.smithi107.stdout:1/421: mkdir d6/d86 0 2024-03-11T19:55:56.468 INFO:tasks.workunit.client.0.smithi107.stdout:1/422: truncate d6/fd 3837917 0 2024-03-11T19:55:56.468 INFO:tasks.workunit.client.0.smithi107.stdout:1/423: creat d6/d28/f87 x:0 0 0 2024-03-11T19:55:56.468 INFO:tasks.workunit.client.0.smithi107.stdout:1/424: write d6/d7e/d81/d27/d2c/d32/f6f [2091159,91688] 0 2024-03-11T19:55:56.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/425: getdents d6/d86 0 2024-03-11T19:55:56.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/383: dread d4/d7/d1f/d2b/d24/d26/d29/f64 [0,4194304] 0 2024-03-11T19:55:56.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/384: mknod d4/d7/d1f/d2b/c71 0 2024-03-11T19:55:56.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/372: dread d1/d10/d13/d4a/f5c [0,4194304] 0 2024-03-11T19:55:56.497 INFO:tasks.workunit.client.0.smithi107.stdout:0/377: dwrite fd [4194304,4194304] 0 2024-03-11T19:55:56.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/426: fdatasync d6/d7e/d81/d27/d2c/d32/f6f 0 2024-03-11T19:55:56.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/427: write d6/d28/f87 [177828,46434] 0 2024-03-11T19:55:56.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/428: mkdir d6/d7e/d88 0 2024-03-11T19:55:56.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/378: stat l2 0 2024-03-11T19:55:56.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/379: chown de/d11/d23/d24/c33 8879687 1 2024-03-11T19:55:56.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/373: rename d1/d19 to d1/d10/d13/d6c 0 2024-03-11T19:55:56.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/374: stat d1/d10/f57 0 2024-03-11T19:55:56.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/375: getdents d1/d10/d4e 0 2024-03-11T19:55:56.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/376: stat d1/d10/d13/d4a/d4c/d49/f54 0 2024-03-11T19:55:56.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/377: dread - d1/d10/d21/d43/d48/f56 zero size 2024-03-11T19:55:56.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/447: read d6/d21/f41 [2662057,47698] 0 2024-03-11T19:55:56.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/448: rmdir d6/da/d16/d5c/d7d/d60 39 2024-03-11T19:55:56.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/449: truncate d6/d9/d4e/f59 1002390 0 2024-03-11T19:55:56.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/450: creat d6/da/d16/d5c/d7d/d60/d61/d78/f8d x:0 0 0 2024-03-11T19:55:56.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/451: truncate d6/da/d16/d5c/f54 4718294 0 2024-03-11T19:55:56.530 INFO:tasks.workunit.client.0.smithi107.stdout:4/385: dread d4/d7/d1f/d2b/d24/d26/d29/f3f [0,4194304] 0 2024-03-11T19:55:56.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/367: dwrite d1/d1a/d1e/d26/f62 [0,4194304] 0 2024-03-11T19:55:56.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/368: fdatasync d1/d1a/d1e/d26/f47 0 2024-03-11T19:55:56.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/429: dwrite d6/d7e/d81/d27/d2c/f30 [0,4194304] 0 2024-03-11T19:55:56.541 INFO:tasks.workunit.client.0.smithi107.stdout:2/369: mkdir d1/d38/d75 0 2024-03-11T19:55:56.541 INFO:tasks.workunit.client.0.smithi107.stdout:4/386: getdents d4/d7/d1f/d2b/d24/d26/d29/d36 0 2024-03-11T19:55:56.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/380: rename de/d11/d23/d32/d34/d39/f5b to de/f75 0 2024-03-11T19:55:56.542 INFO:tasks.workunit.client.0.smithi107.stdout:4/387: readlink d4/d7/d2f/d33/d3a/l61 0 2024-03-11T19:55:56.542 INFO:tasks.workunit.client.0.smithi107.stdout:2/370: mkdir d1/d38/d75/d76 0 2024-03-11T19:55:56.542 INFO:tasks.workunit.client.0.smithi107.stdout:2/371: creat d1/d1d/f77 x:0 0 0 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/372: creat d1/d21/d35/f78 x:0 0 0 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/373: creat d1/d1d/f79 x:0 0 0 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/374: creat d1/d1a/d1e/d26/f7a x:0 0 0 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/375: chown d1/d1a/c50 16 1 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/376: chown d1/d1a/d1e/d41 549328568 1 2024-03-11T19:55:56.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/378: rename d1/d10/d26/d17/f5b to d1/d10/d13/d6c/d2d/f6d 0 2024-03-11T19:55:56.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/377: write d1/d1a/d1e/d26/f30 [259798,35587] 0 2024-03-11T19:55:56.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/379: creat d1/f6e x:0 0 0 2024-03-11T19:55:56.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/380: stat d1/d10/f57 0 2024-03-11T19:55:56.547 INFO:tasks.workunit.client.0.smithi107.stdout:6/340: sync 2024-03-11T19:55:56.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/359: sync 2024-03-11T19:55:56.549 INFO:tasks.workunit.client.0.smithi107.stdout:8/360: truncate dd/de/d1b/d20/d5e/f65 707515 0 2024-03-11T19:55:56.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/341: rename f8 to d10/d19/f74 0 2024-03-11T19:55:56.561 INFO:tasks.workunit.client.0.smithi107.stdout:6/342: creat d10/d19/d38/f75 x:0 0 0 2024-03-11T19:55:56.562 INFO:tasks.workunit.client.0.smithi107.stdout:6/343: mkdir d10/d16/d1f/d51/d76 0 2024-03-11T19:55:56.562 INFO:tasks.workunit.client.0.smithi107.stdout:6/344: creat d10/d19/f77 x:0 0 0 2024-03-11T19:55:56.566 INFO:tasks.workunit.client.0.smithi107.stdout:6/345: symlink d10/d16/l78 0 2024-03-11T19:55:56.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/346: unlink d10/d16/f1c 0 2024-03-11T19:55:56.578 INFO:tasks.workunit.client.0.smithi107.stdout:6/347: unlink d10/d19/l4a 0 2024-03-11T19:55:56.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/430: write d6/d28/f4b [3623027,115414] 0 2024-03-11T19:55:56.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/431: stat d6/d7e/d88 0 2024-03-11T19:55:56.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/381: dwrite de/f44 [0,4194304] 0 2024-03-11T19:55:56.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/382: write de/d11/d23/d32/d34/d39/d3a/f50 [1337833,10604] 0 2024-03-11T19:55:56.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/432: stat d6/d19/c5e 0 2024-03-11T19:55:56.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/348: stat d10/d16 0 2024-03-11T19:55:56.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/349: fdatasync d10/d19/f74 0 2024-03-11T19:55:56.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/452: dwrite d6/f7e [0,4194304] 0 2024-03-11T19:55:56.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/350: creat d10/d16/d1f/f79 x:0 0 0 2024-03-11T19:55:56.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/381: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:56.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/382: write d1/d10/d2a/f38 [4755104,1624] 0 2024-03-11T19:55:56.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/383: write d1/d10/d13/d4a/d4c/d1a/f20 [4365048,118205] 0 2024-03-11T19:55:56.632 INFO:tasks.workunit.client.0.smithi107.stdout:0/383: write de/d11/d23/d32/d34/d39/f5a [2647056,37378] 0 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/361: dwrite dd/de/d49/f2b [0,4194304] 0 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:6/351: rename d10/d3d/d54/d67 to d10/d16/d1f/d51/d7a 0 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:6/352: fdatasync d10/d16/d24/d2e/f55 0 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/388: dwrite d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/389: chown d4/d7/d1f/d2b/c50 524 1 2024-03-11T19:55:56.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/390: chown d4/d7/d1f/d2b/d24/d26/d29 101221072 1 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/391: stat d4/d7/d1f/l39 0 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/392: chown d4/d7/d2f/d33/l3d 17 1 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/393: fdatasync d4/d7/d1f/d2b/d24/d26/d29/f30 0 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/378: dwrite d1/d1a/d34/d4f/d63/f6f [0,4194304] 0 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/379: creat d1/f7b x:0 0 0 2024-03-11T19:55:56.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/380: truncate d1/d1a/f60 947510 0 2024-03-11T19:55:56.635 INFO:tasks.workunit.client.0.smithi107.stdout:2/381: creat d1/d1a/f7c x:0 0 0 2024-03-11T19:55:56.639 INFO:tasks.workunit.client.0.smithi107.stdout:2/382: creat d1/d38/d75/f7d x:0 0 0 2024-03-11T19:55:56.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/353: mknod d10/d16/d2b/d39/d6a/c7b 0 2024-03-11T19:55:56.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/383: creat d1/d21/d35/f7e x:0 0 0 2024-03-11T19:55:56.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/384: fdatasync d1/f15 0 2024-03-11T19:55:56.641 INFO:tasks.workunit.client.0.smithi107.stdout:6/354: write d10/d3d/f4c [231650,30678] 0 2024-03-11T19:55:56.641 INFO:tasks.workunit.client.0.smithi107.stdout:6/355: truncate d10/d19/d38/d60/f69 185034 0 2024-03-11T19:55:56.642 INFO:tasks.workunit.client.0.smithi107.stdout:2/385: truncate d1/fc 2790213 0 2024-03-11T19:55:56.646 INFO:tasks.workunit.client.0.smithi107.stdout:0/384: write de/d11/f35 [8654,30265] 0 2024-03-11T19:55:56.646 INFO:tasks.workunit.client.0.smithi107.stdout:0/385: chown de/d11/d23/d32/f54 6 1 2024-03-11T19:55:56.650 INFO:tasks.workunit.client.0.smithi107.stdout:1/433: dwrite d6/d7e/d81/d27/f47 [4194304,4194304] 0 2024-03-11T19:55:56.650 INFO:tasks.workunit.client.0.smithi107.stdout:1/434: chown d6/d7e/d81/d27/l5d 5 1 2024-03-11T19:55:56.650 INFO:tasks.workunit.client.0.smithi107.stdout:1/435: creat d6/d28/d60/f89 x:0 0 0 2024-03-11T19:55:56.650 INFO:tasks.workunit.client.0.smithi107.stdout:7/453: dwrite d6/da/d16/d5c/f4a [0,4194304] 0 2024-03-11T19:55:56.651 INFO:tasks.workunit.client.0.smithi107.stdout:0/386: rename de/f17 to de/d11/d23/f76 0 2024-03-11T19:55:56.652 INFO:tasks.workunit.client.0.smithi107.stdout:1/436: creat d6/d7e/d81/d27/d2c/d72/f8a x:0 0 0 2024-03-11T19:55:56.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/454: symlink d6/da/l8e 0 2024-03-11T19:55:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:1/437: creat d6/d7e/f8b x:0 0 0 2024-03-11T19:55:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:2/386: write d1/d21/d35/f54 [344269,107869] 0 2024-03-11T19:55:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:1/438: chown d6/d19/d3d/d82/c66 3654909 1 2024-03-11T19:55:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:2/387: truncate d1/f40 5109820 0 2024-03-11T19:55:56.664 INFO:tasks.workunit.client.0.smithi107.stdout:1/439: rename d6/d7e/d81 to d6/d7e/d81/d27/d2c/d32/d8c 22 2024-03-11T19:55:56.667 INFO:tasks.workunit.client.0.smithi107.stdout:2/388: rename d1/d1a/d1e/d41/l57 to d1/d1a/d1e/d55/l7f 0 2024-03-11T19:55:56.671 INFO:tasks.workunit.client.0.smithi107.stdout:6/356: write d10/d16/f33 [3879144,28851] 0 2024-03-11T19:55:56.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/387: rmdir de/d11/d23/d32/d34/d39 39 2024-03-11T19:55:56.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/388: chown de/c30 284310239 1 2024-03-11T19:55:56.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/394: read d4/d7/f17 [763993,86554] 0 2024-03-11T19:55:56.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/395: link d4/d7/d2f/d33/l3d d4/d7/db/d5c/l72 0 2024-03-11T19:55:56.677 INFO:tasks.workunit.client.0.smithi107.stdout:0/389: symlink de/d11/l77 0 2024-03-11T19:55:56.677 INFO:tasks.workunit.client.0.smithi107.stdout:5/468: sync 2024-03-11T19:55:56.677 INFO:tasks.workunit.client.0.smithi107.stdout:0/390: fdatasync de/f75 0 2024-03-11T19:55:56.678 INFO:tasks.workunit.client.0.smithi107.stdout:9/418: sync 2024-03-11T19:55:56.678 INFO:tasks.workunit.client.0.smithi107.stdout:9/419: truncate d4/d1d/d1f/d28/d2c/f49 451075 0 2024-03-11T19:55:56.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/396: creat d4/d7/d2f/d3c/f73 x:0 0 0 2024-03-11T19:55:56.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/384: dwrite d1/d10/d21/d43/d48/f69 [0,4194304] 0 2024-03-11T19:55:56.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/385: chown d1/d45 4485016 1 2024-03-11T19:55:56.679 INFO:tasks.workunit.client.0.smithi107.stdout:5/469: mknod da/d6e/c8f 0 2024-03-11T19:55:56.679 INFO:tasks.workunit.client.0.smithi107.stdout:5/470: readlink da/d14/d18/d1e/l5d 0 2024-03-11T19:55:56.679 INFO:tasks.workunit.client.0.smithi107.stdout:5/471: rename da to da/d14/d43/d47/d90 22 2024-03-11T19:55:56.680 INFO:tasks.workunit.client.0.smithi107.stdout:0/391: unlink de/d11/c2b 0 2024-03-11T19:55:56.682 INFO:tasks.workunit.client.0.smithi107.stdout:4/397: symlink d4/d7/d1f/d2b/d19/l74 0 2024-03-11T19:55:56.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/455: write d6/da/d16/d5c/d7d/d1b/f1c [2290777,40777] 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/456: truncate d6/d9/d57/d65/f6a 1288331 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/457: fdatasync d6/da/f1f 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:4/398: write d4/d7/d2f/f65 [1012720,82863] 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:4/399: stat d4 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/458: creat d6/da/f8f x:0 0 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/459: truncate d6/da/f7c 1251894 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/460: fsync d6/d9/f85 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/461: readlink d6/da/d49/l79 0 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/462: chown d6/da/d16/d5c/d7d/d1b 177160 1 2024-03-11T19:55:56.683 INFO:tasks.workunit.client.0.smithi107.stdout:3/386: truncate d1/d10/d13/d6c/f44 3992830 0 2024-03-11T19:55:56.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/392: creat de/d11/d23/d24/d4f/f78 x:0 0 0 2024-03-11T19:55:56.685 INFO:tasks.workunit.client.0.smithi107.stdout:4/400: mkdir d4/d7/d1f/d2b/d19/d75 0 2024-03-11T19:55:56.686 INFO:tasks.workunit.client.0.smithi107.stdout:7/463: creat d6/d9/d57/d65/f90 x:0 0 0 2024-03-11T19:55:56.689 INFO:tasks.workunit.client.0.smithi107.stdout:3/387: creat d1/d10/f6f x:0 0 0 2024-03-11T19:55:56.689 INFO:tasks.workunit.client.0.smithi107.stdout:7/464: write d6/da/f1f [4971702,1167] 0 2024-03-11T19:55:56.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/393: creat de/d11/d74/f79 x:0 0 0 2024-03-11T19:55:56.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/394: fdatasync de/d11/d23/d26/f48 0 2024-03-11T19:55:56.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/395: chown de/f44 2139487538 1 2024-03-11T19:55:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/472: write da/d14/d18/f2c [82837,48776] 0 2024-03-11T19:55:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/473: dread - da/d14/f79 zero size 2024-03-11T19:55:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/474: chown da/d14/d43/d7f 55137480 1 2024-03-11T19:55:56.698 INFO:tasks.workunit.client.0.smithi107.stdout:5/475: write da/d14/d18/f4c [653525,71988] 0 2024-03-11T19:55:56.701 INFO:tasks.workunit.client.0.smithi107.stdout:4/401: mknod d4/d7/d2f/c76 0 2024-03-11T19:55:56.701 INFO:tasks.workunit.client.0.smithi107.stdout:4/402: creat d4/d7/d2f/d3c/f77 x:0 0 0 2024-03-11T19:55:56.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/362: dwrite dd/de/d1b/d20/f43 [0,4194304] 0 2024-03-11T19:55:56.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/465: read d6/da/d16/d5c/f70 [1057699,122654] 0 2024-03-11T19:55:56.710 INFO:tasks.workunit.client.0.smithi107.stdout:5/476: creat da/d14/d43/d47/f91 x:0 0 0 2024-03-11T19:55:56.711 INFO:tasks.workunit.client.0.smithi107.stdout:8/363: link dd/f69 dd/d13/f6c 0 2024-03-11T19:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/466: mkdir d6/da/d16/d91 0 2024-03-11T19:55:56.717 INFO:tasks.workunit.client.0.smithi107.stdout:5/477: creat da/d14/d38/d59/d54/d73/f92 x:0 0 0 2024-03-11T19:55:56.717 INFO:tasks.workunit.client.0.smithi107.stdout:7/467: creat d6/da/d16/d91/f92 x:0 0 0 2024-03-11T19:55:56.721 INFO:tasks.workunit.client.0.smithi107.stdout:1/440: dwrite d6/d28/f4b [0,4194304] 0 2024-03-11T19:55:56.725 INFO:tasks.workunit.client.0.smithi107.stdout:0/396: read de/d11/d23/d32/d34/d39/f5a [2350420,94282] 0 2024-03-11T19:55:56.725 INFO:tasks.workunit.client.0.smithi107.stdout:0/397: chown de/d58/f61 30485 1 2024-03-11T19:55:56.725 INFO:tasks.workunit.client.0.smithi107.stdout:0/398: readlink de/l1c 0 2024-03-11T19:55:56.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/478: symlink da/d14/d43/l93 0 2024-03-11T19:55:56.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/399: mkdir de/d7a 0 2024-03-11T19:55:56.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/400: dread - de/d11/d23/d32/d34/d39/d3a/f5e zero size 2024-03-11T19:55:56.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/479: symlink da/d14/d38/l94 0 2024-03-11T19:55:56.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/480: creat da/d14/d18/f95 x:0 0 0 2024-03-11T19:55:56.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/481: readlink da/d14/l22 0 2024-03-11T19:55:56.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/403: write d4/d7/d1f/d2b/d24/d26/f43 [4554090,84598] 0 2024-03-11T19:55:56.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/404: fsync d4/f6 0 2024-03-11T19:55:56.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/482: rename da/d14/d77 to da/d14/d96 0 2024-03-11T19:55:56.731 INFO:tasks.workunit.client.0.smithi107.stdout:4/405: creat d4/d7/d1f/d2b/d24/d26/d29/d36/f78 x:0 0 0 2024-03-11T19:55:56.731 INFO:tasks.workunit.client.0.smithi107.stdout:4/406: fsync d4/d7/d1f/d2b/d24/d26/d29/d36/f57 0 2024-03-11T19:55:56.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/483: creat da/d14/d38/d59/d54/f97 x:0 0 0 2024-03-11T19:55:56.733 INFO:tasks.workunit.client.0.smithi107.stdout:4/407: mkdir d4/d7/d2f/d33/d3a/d79 0 2024-03-11T19:55:56.734 INFO:tasks.workunit.client.0.smithi107.stdout:4/408: getdents d4/d7 0 2024-03-11T19:55:56.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/364: getdents dd/de 0 2024-03-11T19:55:56.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/365: write dd/d13/d18/f5c [332683,78033] 0 2024-03-11T19:55:56.750 INFO:tasks.workunit.client.0.smithi107.stdout:4/409: rename d4/d7/d1f/d2b/f67 to d4/d7/d1f/d2b/d24/d26/d29/d36/f7a 0 2024-03-11T19:55:56.750 INFO:tasks.workunit.client.0.smithi107.stdout:4/410: fdatasync d4/d7/d1f/d2b/d24/d26/f51 0 2024-03-11T19:55:56.754 INFO:tasks.workunit.client.0.smithi107.stdout:4/411: mkdir d4/d7/d2f/d33/d7b 0 2024-03-11T19:55:56.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/412: mknod d4/d7/d1f/d2b/d24/c7c 0 2024-03-11T19:55:56.756 INFO:tasks.workunit.client.0.smithi107.stdout:4/413: creat d4/d7/d1f/d2b/d24/d26/d29/d36/f7d x:0 0 0 2024-03-11T19:55:56.756 INFO:tasks.workunit.client.0.smithi107.stdout:4/414: creat d4/d7/d2f/d3b/f7e x:0 0 0 2024-03-11T19:55:56.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/389: dwrite d1/d21/d35/f7e [0,4194304] 0 2024-03-11T19:55:56.758 INFO:tasks.workunit.client.0.smithi107.stdout:4/415: symlink d4/d7/d2f/d33/l7f 0 2024-03-11T19:55:56.759 INFO:tasks.workunit.client.0.smithi107.stdout:0/401: write de/f44 [3083791,26345] 0 2024-03-11T19:55:56.762 INFO:tasks.workunit.client.0.smithi107.stdout:2/390: mkdir d1/d1d/d80 0 2024-03-11T19:55:56.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/420: dwrite d4/f16 [0,4194304] 0 2024-03-11T19:55:56.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/421: creat d4/d1d/d1f/d28/d2c/fa5 x:0 0 0 2024-03-11T19:55:56.765 INFO:tasks.workunit.client.0.smithi107.stdout:4/416: getdents d4/d7/d2f 0 2024-03-11T19:55:56.765 INFO:tasks.workunit.client.0.smithi107.stdout:4/417: truncate d4/d7/d1f/d2b/d19/f70 639591 0 2024-03-11T19:55:56.766 INFO:tasks.workunit.client.0.smithi107.stdout:4/418: write d4/d7/f17 [1991452,53140] 0 2024-03-11T19:55:56.769 INFO:tasks.workunit.client.0.smithi107.stdout:5/484: dread f5 [0,4194304] 0 2024-03-11T19:55:56.769 INFO:tasks.workunit.client.0.smithi107.stdout:5/485: chown da/d14/d38/d46/l53 6371873 1 2024-03-11T19:55:56.770 INFO:tasks.workunit.client.0.smithi107.stdout:0/402: rmdir de/d11/d74 39 2024-03-11T19:55:56.770 INFO:tasks.workunit.client.0.smithi107.stdout:0/403: chown de/d11/d23/d24/l4e 4 1 2024-03-11T19:55:56.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/486: write da/d14/d96/f88 [696899,90215] 0 2024-03-11T19:55:56.770 INFO:tasks.workunit.client.0.smithi107.stdout:6/357: dwrite d10/f13 [0,4194304] 0 2024-03-11T19:55:56.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/404: symlink de/d11/d23/d32/d34/d39/d3a/d72/l7b 0 2024-03-11T19:55:56.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/487: mkdir da/d14/d43/d7f/d98 0 2024-03-11T19:55:56.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/441: dwrite d6/d7e/d81/d27/d2c/d72/f75 [4194304,4194304] 0 2024-03-11T19:55:56.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/442: readlink d6/d7e/d81/l1f 0 2024-03-11T19:55:56.783 INFO:tasks.workunit.client.0.smithi107.stdout:4/419: truncate d4/d7/d1f/d2b/d19/f70 342651 0 2024-03-11T19:55:56.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/488: dread da/d31/f75 [0,4194304] 0 2024-03-11T19:55:56.806 INFO:tasks.workunit.client.0.smithi107.stdout:7/468: dwrite d6/f7e [0,4194304] 0 2024-03-11T19:55:56.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/391: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:55:56.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/489: symlink da/l99 0 2024-03-11T19:55:56.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/392: getdents d1/d1a/d1e/d41 0 2024-03-11T19:55:56.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/393: fdatasync d1/f49 0 2024-03-11T19:55:56.815 INFO:tasks.workunit.client.0.smithi107.stdout:2/394: symlink d1/d1d/l81 0 2024-03-11T19:55:56.815 INFO:tasks.workunit.client.0.smithi107.stdout:2/395: fsync d1/d1a/d1e/d26/f62 0 2024-03-11T19:55:56.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/422: dwrite d4/d1d/d1f/f2f [4194304,4194304] 0 2024-03-11T19:55:56.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/423: link d4/d1d/d1f/d28/d2c/f82 d4/d1d/d5c/fa6 0 2024-03-11T19:55:56.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/405: dwrite de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:56.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/406: write de/f18 [4071867,62066] 0 2024-03-11T19:55:56.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/407: truncate de/d11/f1b 1902533 0 2024-03-11T19:55:56.839 INFO:tasks.workunit.client.0.smithi107.stdout:7/469: dwrite d6/d9/d4e/f59 [0,4194304] 0 2024-03-11T19:55:56.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/490: dwrite da/d14/d38/d59/f64 [0,4194304] 0 2024-03-11T19:55:56.852 INFO:tasks.workunit.client.0.smithi107.stdout:2/396: dwrite d1/d1a/d1e/d26/f47 [0,4194304] 0 2024-03-11T19:55:56.852 INFO:tasks.workunit.client.0.smithi107.stdout:2/397: readlink d1/l5a 0 2024-03-11T19:55:56.852 INFO:tasks.workunit.client.0.smithi107.stdout:2/398: read - d1/d1d/f77 zero size 2024-03-11T19:55:56.856 INFO:tasks.workunit.client.0.smithi107.stdout:5/491: read da/d14/d96/f88 [418087,107425] 0 2024-03-11T19:55:56.856 INFO:tasks.workunit.client.0.smithi107.stdout:9/424: dread d4/d1d/d1f/d28/d2c/d56/d6d/f2d [0,4194304] 0 2024-03-11T19:55:56.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/425: mkdir d4/d9/d9f/da7 0 2024-03-11T19:55:56.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/426: truncate d4/d9/fa 5160226 0 2024-03-11T19:55:56.868 INFO:tasks.workunit.client.0.smithi107.stdout:0/408: dwrite de/d11/f45 [0,4194304] 0 2024-03-11T19:55:56.868 INFO:tasks.workunit.client.0.smithi107.stdout:4/420: rmdir d4/d7/d1f/d2b 39 2024-03-11T19:55:56.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/443: rename d6/d28/f87 to d6/d7e/d81/d27/d2c/d32/f8d 0 2024-03-11T19:55:56.876 INFO:tasks.workunit.client.0.smithi107.stdout:1/444: fdatasync d6/d28/f41 0 2024-03-11T19:55:56.876 INFO:tasks.workunit.client.0.smithi107.stdout:1/445: fsync d6/d7e/d81/d27/f47 0 2024-03-11T19:55:56.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/358: unlink d10/c4d 0 2024-03-11T19:55:56.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/359: chown d10/d16/d24/d2e/f36 4 1 2024-03-11T19:55:56.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/409: mkdir de/d11/d23/d32/d34/d39/d55/d7c 0 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/410: chown de/d11/d23/d32/d34/d39/f6e 345 1 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/421: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/c80 0 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/422: readlink d4/d7/d1f/d2b/d19/l4c 0 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:6/360: rmdir d10/d16/d24/d2e 39 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/492: rename da/d31/d42/f40 to da/d14/d43/f9a 0 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/493: dread - da/d14/d18/f76 zero size 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/494: dread - da/d14/d43/d47/f91 zero size 2024-03-11T19:55:56.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/495: fdatasync da/d14/d43/f6c 0 2024-03-11T19:55:56.879 INFO:tasks.workunit.client.0.smithi107.stdout:6/361: symlink d10/d16/d1f/l7c 0 2024-03-11T19:55:56.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/362: truncate d10/d16/d24/d2e/f34 932957 0 2024-03-11T19:55:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:5/496: rename da/d14/d38/d59/d54 to da/d14/d18/d1e/d9b 0 2024-03-11T19:55:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:5/497: fdatasync da/d6e/f82 0 2024-03-11T19:55:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/411: link de/d11/d74/f79 de/d11/d23/d32/d34/d39/d3a/d72/f7d 0 2024-03-11T19:55:56.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/412: dread - de/d11/d23/d32/d34/f6c zero size 2024-03-11T19:55:56.894 INFO:tasks.workunit.client.0.smithi107.stdout:4/423: dread d4/d7/d1f/d2b/d24/d26/f51 [0,4194304] 0 2024-03-11T19:55:56.894 INFO:tasks.workunit.client.0.smithi107.stdout:4/424: truncate d4/d7/d1f/d2b/d24/d26/d29/d36/f7a 1732850 0 2024-03-11T19:55:56.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/413: rmdir de/d11/d23/d32/d34/d39/d3a/d72 39 2024-03-11T19:55:56.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/414: write f5 [868787,112767] 0 2024-03-11T19:55:56.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/415: truncate de/d11/d23/d26/f29 582894 0 2024-03-11T19:55:56.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/416: read - de/d11/d23/d24/d4f/f78 zero size 2024-03-11T19:55:56.897 INFO:tasks.workunit.client.0.smithi107.stdout:0/417: symlink de/d11/d23/d32/d34/d39/l7e 0 2024-03-11T19:55:56.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/425: dread d4/d7/d1f/d2b/d24/d26/f51 [0,4194304] 0 2024-03-11T19:55:56.911 INFO:tasks.workunit.client.0.smithi107.stdout:4/426: truncate d4/d7/f1d 369296 0 2024-03-11T19:55:56.911 INFO:tasks.workunit.client.0.smithi107.stdout:4/427: creat d4/d7/d2f/d3c/f81 x:0 0 0 2024-03-11T19:55:56.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/428: write d4/d7/d1f/d2b/f56 [4257712,130845] 0 2024-03-11T19:55:56.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/429: write d4/d7/d2f/d3c/f73 [447416,102619] 0 2024-03-11T19:55:56.914 INFO:tasks.workunit.client.0.smithi107.stdout:9/427: dwrite d4/d9/d4d/f8e [0,4194304] 0 2024-03-11T19:55:56.924 INFO:tasks.workunit.client.0.smithi107.stdout:9/428: unlink d4/d1d/d1f/d28/f6c 0 2024-03-11T19:55:56.924 INFO:tasks.workunit.client.0.smithi107.stdout:9/429: chown d4/f54 1189000 1 2024-03-11T19:55:56.925 INFO:tasks.workunit.client.0.smithi107.stdout:4/430: getdents d4/d7/d2f/d33 0 2024-03-11T19:55:56.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/446: dwrite d6/d7e/d81/f5c [4194304,4194304] 0 2024-03-11T19:55:56.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/418: dread de/f2f [0,4194304] 0 2024-03-11T19:55:56.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/419: creat de/d58/f7f x:0 0 0 2024-03-11T19:55:56.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/420: truncate de/d11/d23/d24/f2d 5173756 0 2024-03-11T19:55:56.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/421: fdatasync f5 0 2024-03-11T19:55:56.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/363: dwrite d10/f71 [0,4194304] 0 2024-03-11T19:55:56.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/447: mknod d6/d7e/d88/c8e 0 2024-03-11T19:55:56.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/498: dwrite da/d14/f68 [0,4194304] 0 2024-03-11T19:55:56.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/499: chown da/d14/d43/f6c 1 1 2024-03-11T19:55:56.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/500: rmdir da/d14/d18/d1e 39 2024-03-11T19:55:56.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/501: truncate da/d14/d43/d47/f91 855881 0 2024-03-11T19:55:56.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/502: fsync da/d14/d38/d59/f69 0 2024-03-11T19:55:56.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/364: symlink d10/l7d 0 2024-03-11T19:55:56.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/365: dread - d10/d19/d3b/f5e zero size 2024-03-11T19:55:56.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/366: write d10/d16/d1f/d51/f6c [705486,3433] 0 2024-03-11T19:55:56.946 INFO:tasks.workunit.client.0.smithi107.stdout:5/503: rename da/d14/d38/d59/c3b to da/d14/d43/d7f/c9c 0 2024-03-11T19:55:56.946 INFO:tasks.workunit.client.0.smithi107.stdout:5/504: readlink da/d14/d18/l74 0 2024-03-11T19:55:56.947 INFO:tasks.workunit.client.0.smithi107.stdout:6/367: chown d10/d19/d3b/f4e 913 1 2024-03-11T19:55:56.947 INFO:tasks.workunit.client.0.smithi107.stdout:6/368: fsync d10/d16/f33 0 2024-03-11T19:55:56.948 INFO:tasks.workunit.client.0.smithi107.stdout:6/369: creat d10/d16/d24/d2e/f7e x:0 0 0 2024-03-11T19:55:56.950 INFO:tasks.workunit.client.0.smithi107.stdout:6/370: unlink d10/d16/d2b/d39/d6a/c7b 0 2024-03-11T19:55:56.950 INFO:tasks.workunit.client.0.smithi107.stdout:6/371: fdatasync fc 0 2024-03-11T19:55:56.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/372: mkdir d10/d16/d1f/d51/d76/d7f 0 2024-03-11T19:55:56.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/505: read da/d14/d18/f27 [3033296,39570] 0 2024-03-11T19:55:56.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/506: chown da/d31/f4f 229 1 2024-03-11T19:55:56.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/507: chown da/fb 1578954 1 2024-03-11T19:55:56.973 INFO:tasks.workunit.client.0.smithi107.stdout:9/430: dwrite d4/d1d/d1f/d28/d2c/f82 [0,4194304] 0 2024-03-11T19:55:56.973 INFO:tasks.workunit.client.0.smithi107.stdout:9/431: creat d4/d1d/d5c/d64/fa8 x:0 0 0 2024-03-11T19:55:56.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/432: rmdir d4/d1d/d1f/d28/d5d 39 2024-03-11T19:55:56.990 INFO:tasks.workunit.client.0.smithi107.stdout:4/431: dwrite d4/d7/d1f/d2b/d24/d26/f38 [0,4194304] 0 2024-03-11T19:55:56.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/432: mkdir d4/d7/d1f/d82 0 2024-03-11T19:55:56.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/422: dwrite de/d11/d23/d32/d34/d39/d3a/f50 [0,4194304] 0 2024-03-11T19:55:56.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/433: creat d4/d7/d1f/d82/f83 x:0 0 0 2024-03-11T19:55:56.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/423: symlink de/d11/d23/d32/l80 0 2024-03-11T19:55:56.997 INFO:tasks.workunit.client.0.smithi107.stdout:4/434: creat d4/d7/d1f/d2b/d24/d26/f84 x:0 0 0 2024-03-11T19:55:56.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/508: dread da/d31/f75 [0,4194304] 0 2024-03-11T19:55:56.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/509: fdatasync da/d14/d43/d47/f91 0 2024-03-11T19:55:56.999 INFO:tasks.workunit.client.0.smithi107.stdout:0/424: mknod de/d11/d23/c81 0 2024-03-11T19:55:56.999 INFO:tasks.workunit.client.0.smithi107.stdout:4/435: mknod d4/d7/c85 0 2024-03-11T19:55:57.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/425: write de/d11/d23/d24/f69 [292590,123569] 0 2024-03-11T19:55:57.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/426: creat de/d11/d23/d32/d34/f82 x:0 0 0 2024-03-11T19:55:57.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/427: truncate de/d11/d23/d32/f54 4752391 0 2024-03-11T19:55:57.002 INFO:tasks.workunit.client.0.smithi107.stdout:5/510: truncate da/d14/f5e 7278142 0 2024-03-11T19:55:57.003 INFO:tasks.workunit.client.0.smithi107.stdout:4/436: mknod d4/d7/d1f/d2b/c86 0 2024-03-11T19:55:57.005 INFO:tasks.workunit.client.0.smithi107.stdout:0/428: mkdir de/d11/d23/d24/d83 0 2024-03-11T19:55:57.005 INFO:tasks.workunit.client.0.smithi107.stdout:0/429: write de/d11/d23/d32/d34/f82 [133417,86573] 0 2024-03-11T19:55:57.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/511: mknod da/d14/d18/c9d 0 2024-03-11T19:55:57.006 INFO:tasks.workunit.client.0.smithi107.stdout:6/373: dwrite d10/d19/f77 [0,4194304] 0 2024-03-11T19:55:57.006 INFO:tasks.workunit.client.0.smithi107.stdout:1/448: dwrite d6/d28/d60/f83 [0,4194304] 0 2024-03-11T19:55:57.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/437: symlink d4/d7/d1f/l87 0 2024-03-11T19:55:57.008 INFO:tasks.workunit.client.0.smithi107.stdout:0/430: mknod de/d11/d23/d26/c84 0 2024-03-11T19:55:57.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/374: creat d10/d3d/d54/f80 x:0 0 0 2024-03-11T19:55:57.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/512: mkdir da/d14/d43/d9e 0 2024-03-11T19:55:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:5/513: truncate da/d14/d18/f76 505284 0 2024-03-11T19:55:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/449: unlink d6/d7e/d81/d27/d2c/d72/l77 0 2024-03-11T19:55:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/450: creat d6/d19/d3d/d62/f8f x:0 0 0 2024-03-11T19:55:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/451: chown d6/d7e/d81/d27/d2c/f6a 1001279 1 2024-03-11T19:55:57.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/452: chown d6/d19/d3d/d62/f65 2 1 2024-03-11T19:55:57.015 INFO:tasks.workunit.client.0.smithi107.stdout:1/453: symlink d6/d7e/d81/d27/d2c/d7d/l90 0 2024-03-11T19:55:57.015 INFO:tasks.workunit.client.0.smithi107.stdout:1/454: truncate d6/d28/f7c 882993 0 2024-03-11T19:55:57.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/438: rename d4/f6 to d4/d7/d2f/d33/f88 0 2024-03-11T19:55:57.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/431: rename de/f75 to de/d7a/f85 0 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/439: write d4/fc [802052,45789] 0 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/388: sync 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/366: sync 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/367: dread - dd/de/f4e zero size 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/368: fsync dd/f69 0 2024-03-11T19:55:57.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/440: getdents d4/d7/db/d5e 0 2024-03-11T19:55:57.030 INFO:tasks.workunit.client.0.smithi107.stdout:3/389: creat d1/d10/d13/d6c/d5d/f70 x:0 0 0 2024-03-11T19:55:57.032 INFO:tasks.workunit.client.0.smithi107.stdout:8/369: mknod dd/d13/c6d 0 2024-03-11T19:55:57.033 INFO:tasks.workunit.client.0.smithi107.stdout:8/370: write dd/de/d1b/d20/d5e/f65 [297244,99672] 0 2024-03-11T19:55:57.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/390: rename d1/d10/d13/d4a/f5c to d1/d10/d13/d6c/d2d/d4d/f71 0 2024-03-11T19:55:57.035 INFO:tasks.workunit.client.0.smithi107.stdout:3/391: creat d1/d10/d13/d6c/d2d/f72 x:0 0 0 2024-03-11T19:55:57.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/392: truncate d1/d10/d13/d6c/d2d/d4d/f53 222288 0 2024-03-11T19:55:57.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/393: getdents d1/d10/d26/d17 0 2024-03-11T19:55:57.051 INFO:tasks.workunit.client.0.smithi107.stdout:0/432: dread de/d11/d23/f71 [0,4194304] 0 2024-03-11T19:55:57.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/394: rename d1/d10/d13/d4a/d4c/d1a to d1/d10/d4e/d73 0 2024-03-11T19:55:57.056 INFO:tasks.workunit.client.0.smithi107.stdout:5/514: dwrite da/d14/d38/d59/f1d [0,4194304] 0 2024-03-11T19:55:57.056 INFO:tasks.workunit.client.0.smithi107.stdout:5/515: chown da/d14/d38/d59/c29 42496 1 2024-03-11T19:55:57.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/433: mknod de/d11/d74/c86 0 2024-03-11T19:55:57.059 INFO:tasks.workunit.client.0.smithi107.stdout:5/516: creat da/d14/d38/d46/f9f x:0 0 0 2024-03-11T19:55:57.060 INFO:tasks.workunit.client.0.smithi107.stdout:0/434: symlink de/d11/d23/d26/l87 0 2024-03-11T19:55:57.061 INFO:tasks.workunit.client.0.smithi107.stdout:0/435: fdatasync de/d11/d23/d24/f4a 0 2024-03-11T19:55:57.061 INFO:tasks.workunit.client.0.smithi107.stdout:0/436: stat de/l20 0 2024-03-11T19:55:57.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/517: mknod da/d6e/ca0 0 2024-03-11T19:55:57.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/518: unlink da/d14/d18/d1e/c3c 0 2024-03-11T19:55:57.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/371: dwrite dd/d13/d32/d51/f61 [0,4194304] 0 2024-03-11T19:55:57.065 INFO:tasks.workunit.client.0.smithi107.stdout:6/375: dwrite d10/d16/f33 [0,4194304] 0 2024-03-11T19:55:57.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/519: truncate da/d14/d38/d59/d5f/f6a 2037540 0 2024-03-11T19:55:57.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/520: creat da/d14/d38/d46/fa1 x:0 0 0 2024-03-11T19:55:57.065 INFO:tasks.workunit.client.0.smithi107.stdout:1/455: dwrite d6/d28/d60/f89 [0,4194304] 0 2024-03-11T19:55:57.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/372: truncate dd/d13/d18/f37 4125601 0 2024-03-11T19:55:57.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/456: creat d6/d7e/d88/f91 x:0 0 0 2024-03-11T19:55:57.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/457: chown d6/f55 432288 1 2024-03-11T19:55:57.076 INFO:tasks.workunit.client.0.smithi107.stdout:3/395: write d1/d10/d13/d4a/d4c/f2f [3403053,121364] 0 2024-03-11T19:55:57.077 INFO:tasks.workunit.client.0.smithi107.stdout:5/521: truncate da/d14/d38/d59/f1d 1496771 0 2024-03-11T19:55:57.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/373: dread dd/d16/f1e [0,4194304] 0 2024-03-11T19:55:57.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/374: truncate dd/de/d1b/d20/d5e/f65 1146296 0 2024-03-11T19:55:57.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/375: dread - dd/de/f48 zero size 2024-03-11T19:55:57.079 INFO:tasks.workunit.client.0.smithi107.stdout:3/396: symlink d1/d10/d21/d43/l74 0 2024-03-11T19:55:57.079 INFO:tasks.workunit.client.0.smithi107.stdout:3/397: stat d1/d10/d26/d17 0 2024-03-11T19:55:57.080 INFO:tasks.workunit.client.0.smithi107.stdout:5/522: link da/d31/f75 da/d14/d43/d47/fa2 0 2024-03-11T19:55:57.082 INFO:tasks.workunit.client.0.smithi107.stdout:7/470: sync 2024-03-11T19:55:57.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/471: rename d6/d21/f86 to d6/da/d16/d5c/d7d/d60/f93 0 2024-03-11T19:55:57.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/472: link d6/da/d16/l5b d6/da/d16/d5c/d7d/d60/d61/l94 0 2024-03-11T19:55:57.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/473: stat d6/da/d4d 0 2024-03-11T19:55:57.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/474: write d6/da/d16/f6f [1015988,73856] 0 2024-03-11T19:55:57.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/475: write d6/da/d16/d5c/d7d/f5e [945953,83261] 0 2024-03-11T19:55:57.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/476: symlink d6/da/l95 0 2024-03-11T19:55:57.090 INFO:tasks.workunit.client.0.smithi107.stdout:6/376: write d10/d16/d24/d2e/f55 [6263923,100112] 0 2024-03-11T19:55:57.106 INFO:tasks.workunit.client.0.smithi107.stdout:3/398: dread d1/d10/f57 [0,4194304] 0 2024-03-11T19:55:57.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/399: rename d1/d10/d13/d6c/d2d/f72 to d1/d45/f75 0 2024-03-11T19:55:57.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/377: read d10/d16/f33 [215563,126583] 0 2024-03-11T19:55:57.115 INFO:tasks.workunit.client.0.smithi107.stdout:8/376: dwrite dd/de/d1b/f1d [0,4194304] 0 2024-03-11T19:55:57.118 INFO:tasks.workunit.client.0.smithi107.stdout:8/377: symlink dd/d13/d32/l6e 0 2024-03-11T19:55:57.130 INFO:tasks.workunit.client.0.smithi107.stdout:5/523: dwrite da/d14/d18/f27 [4194304,4194304] 0 2024-03-11T19:55:57.135 INFO:tasks.workunit.client.0.smithi107.stdout:7/477: write d6/da/f1f [3154231,44547] 0 2024-03-11T19:55:57.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/437: dwrite de/d11/d23/d32/d34/d39/d3a/f5e [0,4194304] 0 2024-03-11T19:55:57.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/438: chown de/d11/d23/d32/d34/f36 1913948 1 2024-03-11T19:55:57.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/439: truncate de/d11/f45 4512922 0 2024-03-11T19:55:57.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/440: write de/d11/d23/d32/d34/f82 [751171,65430] 0 2024-03-11T19:55:57.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/441: chown de/d7a/f85 1454142 1 2024-03-11T19:55:57.137 INFO:tasks.workunit.client.0.smithi107.stdout:7/478: truncate d6/da/d16/d5c/f70 1441263 0 2024-03-11T19:55:57.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/442: creat de/f88 x:0 0 0 2024-03-11T19:55:57.138 INFO:tasks.workunit.client.0.smithi107.stdout:3/400: dread d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:55:57.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/479: link d6/d9/d4e/f59 d6/da/d4d/f96 0 2024-03-11T19:55:57.139 INFO:tasks.workunit.client.0.smithi107.stdout:0/443: stat de/d11/d23/d32/d34/d39/d3a/d72/f7d 0 2024-03-11T19:55:57.141 INFO:tasks.workunit.client.0.smithi107.stdout:3/401: mkdir d1/d10/d13/d6c/d5d/d76 0 2024-03-11T19:55:57.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/480: unlink d6/da/d16/d5c/d7d/f67 0 2024-03-11T19:55:57.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/481: truncate d6/da/d16/d5c/d7d/f5e 1365115 0 2024-03-11T19:55:57.150 INFO:tasks.workunit.client.0.smithi107.stdout:6/378: read d10/d16/d24/f44 [5682857,87617] 0 2024-03-11T19:55:57.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/379: chown d10/d16/d2b/d39/l49 17760397 1 2024-03-11T19:55:57.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/380: write d10/d16/d24/f5b [967570,120977] 0 2024-03-11T19:55:57.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/378: write dd/de/d1b/d20/f26 [1479022,121853] 0 2024-03-11T19:55:57.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/379: symlink dd/de/d49/d46/l6f 0 2024-03-11T19:55:57.162 INFO:tasks.workunit.client.0.smithi107.stdout:0/444: rmdir de/d58 39 2024-03-11T19:55:57.163 INFO:tasks.workunit.client.0.smithi107.stdout:6/381: link d10/d19/d38/l4b d10/d16/d2b/d39/l81 0 2024-03-11T19:55:57.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/482: getdents d6/da/d49 0 2024-03-11T19:55:57.164 INFO:tasks.workunit.client.0.smithi107.stdout:0/445: mkdir de/d11/d23/d24/d4f/d89 0 2024-03-11T19:55:57.164 INFO:tasks.workunit.client.0.smithi107.stdout:0/446: chown de/d11/d23/c56 633712 1 2024-03-11T19:55:57.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/402: dread d1/f5 [0,4194304] 0 2024-03-11T19:55:57.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/403: fdatasync d1/d10/d13/d6c/d5d/f70 0 2024-03-11T19:55:57.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/404: chown d1/d10/d21/l5e 40 1 2024-03-11T19:55:57.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/447: truncate fd 2308369 0 2024-03-11T19:55:57.172 INFO:tasks.workunit.client.0.smithi107.stdout:3/405: symlink d1/d10/d13/d6c/d5d/l77 0 2024-03-11T19:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/448: creat de/d11/d23/d32/d34/d39/d3a/d72/f8a x:0 0 0 2024-03-11T19:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/449: fsync de/d11/d23/d26/f48 0 2024-03-11T19:55:57.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/406: mkdir d1/d10/d4e/d78 0 2024-03-11T19:55:57.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/524: dwrite da/d14/d18/f76 [0,4194304] 0 2024-03-11T19:55:57.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/450: creat de/d58/f8b x:0 0 0 2024-03-11T19:55:57.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/483: dread d6/f2f [0,4194304] 0 2024-03-11T19:55:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/484: mkdir d6/da/d16/d91/d97 0 2024-03-11T19:55:57.195 INFO:tasks.workunit.client.0.smithi107.stdout:7/485: rmdir d6/d9 39 2024-03-11T19:55:57.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/486: rename d6/da/d16/d5c/f68 to d6/da/d16/f98 0 2024-03-11T19:55:57.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/487: rename d6/da/d16/d91/f92 to d6/d9/d57/d65/f99 0 2024-03-11T19:55:57.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/380: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:57.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/381: mknod dd/de/d1b/d20/c70 0 2024-03-11T19:55:57.210 INFO:tasks.workunit.client.0.smithi107.stdout:8/382: write dd/de/d49/d46/f4f [588214,36577] 0 2024-03-11T19:55:57.213 INFO:tasks.workunit.client.0.smithi107.stdout:8/383: symlink dd/d13/d32/d51/l71 0 2024-03-11T19:55:57.214 INFO:tasks.workunit.client.0.smithi107.stdout:8/384: symlink dd/d16/l72 0 2024-03-11T19:55:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/382: dwrite d10/d16/d1f/f25 [4194304,4194304] 0 2024-03-11T19:55:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/383: write d10/d19/d38/f5c [519518,11113] 0 2024-03-11T19:55:57.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/384: chown d10/d19/l52 3 1 2024-03-11T19:55:57.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/385: stat fa 0 2024-03-11T19:55:57.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/386: mknod d10/d16/d2b/d39/c82 0 2024-03-11T19:55:57.226 INFO:tasks.workunit.client.0.smithi107.stdout:3/407: dwrite d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:57.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/525: dread da/d14/d18/f1c [0,4194304] 0 2024-03-11T19:55:57.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/526: fsync da/d14/d38/d59/d5f/f80 0 2024-03-11T19:55:57.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/527: stat da/d14/d18/d1e/d9b/c8c 0 2024-03-11T19:55:57.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/528: write da/f85 [191246,113235] 0 2024-03-11T19:55:57.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/529: write da/d14/d38/d46/fa1 [10181,75478] 0 2024-03-11T19:55:57.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/530: rmdir da/d14/d18/d1e/d9b/d73 39 2024-03-11T19:55:57.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/408: rename d1/d10/d13/d6c/d5d/l65 to d1/d10/d13/d4a/d4c/d49/l79 0 2024-03-11T19:55:57.230 INFO:tasks.workunit.client.0.smithi107.stdout:3/409: write d1/d10/d13/d6c/f32 [1168296,19012] 0 2024-03-11T19:55:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/410: symlink d1/d10/d21/d43/l7a 0 2024-03-11T19:55:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:5/531: creat da/d14/d18/d1e/d9b/d73/fa3 x:0 0 0 2024-03-11T19:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/411: creat d1/d10/d13/d4a/d4c/f7b x:0 0 0 2024-03-11T19:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/412: chown d1/d10/d13/d6c/d5d/c66 970409 1 2024-03-11T19:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout:8/385: link dd/d13/f6c dd/de/d49/f73 0 2024-03-11T19:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/399: sync 2024-03-11T19:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/400: truncate d1/d1a/d34/d4f/f5d 298712 0 2024-03-11T19:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/401: readlink d1/d1a/d34/d4f/l53 0 2024-03-11T19:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/402: fdatasync d1/d1a/d1e/d26/f47 0 2024-03-11T19:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout:8/386: unlink dd/de/d1b/d20/f26 0 2024-03-11T19:55:57.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/387: creat dd/de/d1b/d4b/f74 x:0 0 0 2024-03-11T19:55:57.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/403: creat d1/d1a/d34/d6a/f82 x:0 0 0 2024-03-11T19:55:57.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/388: creat dd/d13/d18/f75 x:0 0 0 2024-03-11T19:55:57.240 INFO:tasks.workunit.client.0.smithi107.stdout:8/389: fsync dd/d13/d18/f1a 0 2024-03-11T19:55:57.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/404: write d1/d1d/f79 [658816,62554] 0 2024-03-11T19:55:57.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/413: write d1/d10/d2a/f38 [4485639,98774] 0 2024-03-11T19:55:57.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/390: rename dd/d13/d32/l39 to dd/d13/d32/d51/l76 0 2024-03-11T19:55:57.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/391: write dd/de/d49/d46/f4f [952607,63822] 0 2024-03-11T19:55:57.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/451: dwrite de/d11/f35 [0,4194304] 0 2024-03-11T19:55:57.246 INFO:tasks.workunit.client.0.smithi107.stdout:0/452: write de/d11/d23/d24/d4f/f78 [303750,103744] 0 2024-03-11T19:55:57.246 INFO:tasks.workunit.client.0.smithi107.stdout:3/414: link d1/d10/d26/d17/c46 d1/d10/d13/d4a/d4c/c7c 0 2024-03-11T19:55:57.246 INFO:tasks.workunit.client.0.smithi107.stdout:3/415: chown d1/d10/d4e 26740 1 2024-03-11T19:55:57.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/416: mknod d1/d10/d13/d4a/d4c/c7d 0 2024-03-11T19:55:57.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/417: rmdir d1/d10/d4e/d78 0 2024-03-11T19:55:57.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/418: mkdir d1/d10/d2a/d7e 0 2024-03-11T19:55:57.257 INFO:tasks.workunit.client.0.smithi107.stdout:0/453: rmdir de/d11/d23/d32/d34/d39/d55 39 2024-03-11T19:55:57.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/454: symlink de/d7a/l8c 0 2024-03-11T19:55:57.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/433: sync 2024-03-11T19:55:57.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/441: sync 2024-03-11T19:55:57.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/458: sync 2024-03-11T19:55:57.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/459: mknod d6/d7e/d81/d27/d2c/c92 0 2024-03-11T19:55:57.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/460: dread - d6/d5b/f78 zero size 2024-03-11T19:55:57.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/461: stat d6/d7e/d81/d27/d2c/d32/l70 0 2024-03-11T19:55:57.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/462: read - d6/d7e/d88/f91 zero size 2024-03-11T19:55:57.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/463: fdatasync d6/d7e/d81/d27/f38 0 2024-03-11T19:55:57.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/464: write d6/d7e/d81/d27/d2c/d72/f8a [878327,112048] 0 2024-03-11T19:55:57.268 INFO:tasks.workunit.client.0.smithi107.stdout:1/465: symlink d6/l93 0 2024-03-11T19:55:57.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/466: truncate d6/f1b 373240 0 2024-03-11T19:55:57.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/455: dread de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:55:57.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/442: rmdir d4/d7/d2f/d33/d3a 39 2024-03-11T19:55:57.269 INFO:tasks.workunit.client.0.smithi107.stdout:4/443: stat d4/d7/d1f/d2b/d19/c1e 0 2024-03-11T19:55:57.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/467: rmdir d6/d7e/d81 39 2024-03-11T19:55:57.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/468: chown d6/d7e/d81/f21 2136446 1 2024-03-11T19:55:57.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/469: symlink d6/d5b/l94 0 2024-03-11T19:55:57.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/470: chown d6/d7e/d81/d27/f7a 3347805 1 2024-03-11T19:55:57.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/456: mkdir de/d11/d23/d24/d8d 0 2024-03-11T19:55:57.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/444: truncate d4/d7/db/f21 3706360 0 2024-03-11T19:55:57.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/471: mkdir d6/d28/d60/d95 0 2024-03-11T19:55:57.280 INFO:tasks.workunit.client.0.smithi107.stdout:7/488: dwrite d6/da/d16/f36 [0,4194304] 0 2024-03-11T19:55:57.281 INFO:tasks.workunit.client.0.smithi107.stdout:0/457: symlink de/d7a/l8e 0 2024-03-11T19:55:57.281 INFO:tasks.workunit.client.0.smithi107.stdout:9/434: dread d4/d1d/d1f/d28/f42 [0,4194304] 0 2024-03-11T19:55:57.282 INFO:tasks.workunit.client.0.smithi107.stdout:9/435: stat d4/d9 0 2024-03-11T19:55:57.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/489: truncate d6/d9/d57/d65/f74 1202067 0 2024-03-11T19:55:57.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/436: creat d4/fa9 x:0 0 0 2024-03-11T19:55:57.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/437: truncate d4/d1d/d1f/f39 948373 0 2024-03-11T19:55:57.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/458: unlink de/d11/d23/d32/d34/d39/d3a/d72/l7b 0 2024-03-11T19:55:57.287 INFO:tasks.workunit.client.0.smithi107.stdout:0/459: chown f3 243959478 1 2024-03-11T19:55:57.287 INFO:tasks.workunit.client.0.smithi107.stdout:7/490: rename d6/da/d49/c56 to d6/da/d16/d5c/c9a 0 2024-03-11T19:55:57.289 INFO:tasks.workunit.client.0.smithi107.stdout:0/460: creat de/d11/d23/d32/f8f x:0 0 0 2024-03-11T19:55:57.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/491: mkdir d6/da/d16/d91/d97/d9b 0 2024-03-11T19:55:57.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/492: chown d6/d9/f6d 0 1 2024-03-11T19:55:57.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/493: read - d6/da/d16/d5c/d7d/d60/f93 zero size 2024-03-11T19:55:57.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/445: dread d4/d7/f17 [0,4194304] 0 2024-03-11T19:55:57.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/446: fsync d4/d7/d2f/d3c/f77 0 2024-03-11T19:55:57.298 INFO:tasks.workunit.client.0.smithi107.stdout:2/405: dwrite d1/d21/d35/f78 [0,4194304] 0 2024-03-11T19:55:57.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/406: creat d1/d1a/d34/d4f/f83 x:0 0 0 2024-03-11T19:55:57.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/494: truncate d6/f17 1559122 0 2024-03-11T19:55:57.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/387: dwrite d10/d16/d1f/f42 [0,4194304] 0 2024-03-11T19:55:57.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/388: getdents d10/d19/d38 0 2024-03-11T19:55:57.309 INFO:tasks.workunit.client.0.smithi107.stdout:0/461: link de/d11/d23/d32/d34/d39/d3a/c6b de/d11/d23/d24/c90 0 2024-03-11T19:55:57.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/389: creat d10/d16/d2b/d39/d6a/f83 x:0 0 0 2024-03-11T19:55:57.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/390: chown d10/d16/d24/f44 713019 1 2024-03-11T19:55:57.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/462: creat de/d11/f91 x:0 0 0 2024-03-11T19:55:57.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/463: truncate de/d11/f1b 1657680 0 2024-03-11T19:55:57.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/447: dread d4/d7/d2f/f65 [0,4194304] 0 2024-03-11T19:55:57.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/448: write d4/d7/d1f/f52 [54969,43487] 0 2024-03-11T19:55:57.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/449: chown f3 28170459 1 2024-03-11T19:55:57.324 INFO:tasks.workunit.client.0.smithi107.stdout:1/472: write d6/d7e/d81/d27/d2c/f45 [2092718,126622] 0 2024-03-11T19:55:57.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/419: dwrite d1/d10/d13/f61 [0,4194304] 0 2024-03-11T19:55:57.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/420: rename d1/d10/d4e to d1/d10/d4e/d73/d6b/d7f 22 2024-03-11T19:55:57.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/473: link d6/fd d6/d28/d60/f96 0 2024-03-11T19:55:57.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/464: rmdir de/d11/d23/d32/d34/d39 39 2024-03-11T19:55:57.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/465: fdatasync de/d11/d23/d32/f54 0 2024-03-11T19:55:57.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/438: dread d4/d1d/d5c/fa6 [0,4194304] 0 2024-03-11T19:55:57.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/439: fsync d4/fe 0 2024-03-11T19:55:57.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/440: chown d4/d1d/d1f/d28/d2c/f46 3 1 2024-03-11T19:55:57.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/474: link d6/d7e/d81/l2a d6/d7e/d81/l97 0 2024-03-11T19:55:57.335 INFO:tasks.workunit.client.0.smithi107.stdout:3/421: read d1/d10/d13/d6c/f32 [794135,25047] 0 2024-03-11T19:55:57.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/422: mknod d1/d10/d13/d4a/d4c/c80 0 2024-03-11T19:55:57.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/466: symlink de/d11/l92 0 2024-03-11T19:55:57.341 INFO:tasks.workunit.client.0.smithi107.stdout:3/423: creat d1/d10/d26/d17/f81 x:0 0 0 2024-03-11T19:55:57.350 INFO:tasks.workunit.client.0.smithi107.stdout:2/407: dwrite d1/fc [0,4194304] 0 2024-03-11T19:55:57.351 INFO:tasks.workunit.client.0.smithi107.stdout:9/441: dread d4/d1d/d1f/d28/d2c/d96/f7a [0,4194304] 0 2024-03-11T19:55:57.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/442: unlink d4/f16 0 2024-03-11T19:55:57.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/443: chown d4/d1c/d37/f8f 1 1 2024-03-11T19:55:57.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/444: creat d4/d1d/d5c/d97/faa x:0 0 0 2024-03-11T19:55:57.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/445: chown d4/d1d/d5c/d64/f84 12558440 1 2024-03-11T19:55:57.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/467: dread de/d11/d23/f76 [0,4194304] 0 2024-03-11T19:55:57.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/424: dwrite d1/d10/d13/f61 [4194304,4194304] 0 2024-03-11T19:55:57.379 INFO:tasks.workunit.client.0.smithi107.stdout:3/425: mkdir d1/d10/d21/d43/d82 0 2024-03-11T19:55:57.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/426: mknod d1/d10/d13/d4a/d4c/d49/c83 0 2024-03-11T19:55:57.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/427: write d1/d10/d4e/d73/f20 [3868835,104082] 0 2024-03-11T19:55:57.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/450: dwrite d4/d7/d1f/f20 [0,4194304] 0 2024-03-11T19:55:57.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/428: dread d1/d10/d13/d6c/f32 [0,4194304] 0 2024-03-11T19:55:57.394 INFO:tasks.workunit.client.0.smithi107.stdout:3/429: creat d1/d10/d21/d43/f84 x:0 0 0 2024-03-11T19:55:57.394 INFO:tasks.workunit.client.0.smithi107.stdout:3/430: write d1/f5 [8640567,107109] 0 2024-03-11T19:55:57.399 INFO:tasks.workunit.client.0.smithi107.stdout:6/391: dwrite d10/d16/d24/f5b [0,4194304] 0 2024-03-11T19:55:57.400 INFO:tasks.workunit.client.0.smithi107.stdout:0/468: write de/d11/d23/d24/f4a [3162838,64246] 0 2024-03-11T19:55:57.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/408: dread d1/d21/d35/f7e [0,4194304] 0 2024-03-11T19:55:57.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/409: fdatasync d1/d1a/f74 0 2024-03-11T19:55:57.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/410: link d1/d1a/d1e/d26/f68 d1/d38/d75/f84 0 2024-03-11T19:55:57.411 INFO:tasks.workunit.client.0.smithi107.stdout:2/411: truncate d1/d1a/d1e/d41/f51 1191332 0 2024-03-11T19:55:57.411 INFO:tasks.workunit.client.0.smithi107.stdout:2/412: mknod d1/d38/d75/d76/c85 0 2024-03-11T19:55:57.413 INFO:tasks.workunit.client.0.smithi107.stdout:1/475: dwrite d6/d7e/d88/f91 [0,4194304] 0 2024-03-11T19:55:57.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/413: unlink d1/d1d/c3f 0 2024-03-11T19:55:57.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/414: rmdir d1/d1a/d1e/d26 39 2024-03-11T19:55:57.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/415: symlink d1/d38/d75/l86 0 2024-03-11T19:55:57.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/416: truncate d1/f2b 4150817 0 2024-03-11T19:55:57.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/417: stat d1/d21/c4c 0 2024-03-11T19:55:57.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/418: chown d1/d1a/d34/f42 8 1 2024-03-11T19:55:57.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/419: link d1/d1a/f60 d1/d1a/d1e/d55/f87 0 2024-03-11T19:55:57.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/420: mknod d1/d38/d75/d76/c88 0 2024-03-11T19:55:57.433 INFO:tasks.workunit.client.0.smithi107.stdout:6/392: rename fa to d10/d16/f84 0 2024-03-11T19:55:57.434 INFO:tasks.workunit.client.0.smithi107.stdout:1/476: dread d6/d7e/d81/f29 [0,4194304] 0 2024-03-11T19:55:57.434 INFO:tasks.workunit.client.0.smithi107.stdout:6/393: mkdir d10/d16/d24/d85 0 2024-03-11T19:55:57.434 INFO:tasks.workunit.client.0.smithi107.stdout:6/394: stat d10/d16/d2b/d39/l81 0 2024-03-11T19:55:57.437 INFO:tasks.workunit.client.0.smithi107.stdout:6/395: chown d10/d16/d1f/f28 6 1 2024-03-11T19:55:57.437 INFO:tasks.workunit.client.0.smithi107.stdout:6/396: truncate d10/d3d/f4c 1262179 0 2024-03-11T19:55:57.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/431: dwrite d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:55:57.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/432: fsync d1/d10/d13/d4a/d4c/f2f 0 2024-03-11T19:55:57.448 INFO:tasks.workunit.client.0.smithi107.stdout:2/421: dread d1/d21/d35/f7e [0,4194304] 0 2024-03-11T19:55:57.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/451: dwrite d4/d7/d1f/d2b/d24/f3e [0,4194304] 0 2024-03-11T19:55:57.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/397: link d10/f13 d10/d16/d24/d2e/f86 0 2024-03-11T19:55:57.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/398: readlink d10/d16/d2b/d39/l81 0 2024-03-11T19:55:57.450 INFO:tasks.workunit.client.0.smithi107.stdout:6/399: fdatasync d10/f53 0 2024-03-11T19:55:57.450 INFO:tasks.workunit.client.0.smithi107.stdout:6/400: creat d10/d16/d1f/d51/f87 x:0 0 0 2024-03-11T19:55:57.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/469: rename de/d11/d23/d24/c27 to de/d11/d23/d32/d34/d39/c93 0 2024-03-11T19:55:57.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/401: creat d10/d16/d1f/d51/d76/f88 x:0 0 0 2024-03-11T19:55:57.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/402: chown d10/d19/l35 3 1 2024-03-11T19:55:57.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/452: read d4/d7/d2f/d33/f66 [723103,31690] 0 2024-03-11T19:55:57.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/477: write d6/d7e/d81/f21 [1778005,102353] 0 2024-03-11T19:55:57.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/422: rename d1/d21/d35/f7e to d1/d38/d75/f89 0 2024-03-11T19:55:57.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/470: symlink de/d11/d23/d26/l94 0 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/471: chown de/f40 251016 1 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:5/532: sync 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/423: creat d1/d1d/d64/f8a x:0 0 0 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:5/533: creat da/d14/d18/d1e/d9b/fa4 x:0 0 0 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/472: creat de/d11/d23/d32/d34/d39/d55/f95 x:0 0 0 2024-03-11T19:55:57.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/473: write de/d11/d23/d32/d34/d39/d3a/d72/f8a [386575,25257] 0 2024-03-11T19:55:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/534: mkdir da/d14/d18/d44/da5 0 2024-03-11T19:55:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/535: truncate da/d14/d43/f9a 742363 0 2024-03-11T19:55:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/424: truncate d1/fc 2202601 0 2024-03-11T19:55:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/425: stat d1/d1d/f77 0 2024-03-11T19:55:57.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/536: getdents da/d14/d43/d7f 0 2024-03-11T19:55:57.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/537: truncate da/d14/d43/d47/f91 1048612 0 2024-03-11T19:55:57.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/538: truncate da/d14/d38/d46/fa1 1041999 0 2024-03-11T19:55:57.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/539: creat da/d14/d38/fa6 x:0 0 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/426: getdents d1/d1d/d64 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/433: dread d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/427: link d1/d1a/d1e/d41/l44 d1/d1a/d34/d4f/d63/l8b 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/428: write d1/f49 [2983525,114513] 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/434: mkdir d1/d45/d85 0 2024-03-11T19:55:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/435: fdatasync d1/d10/d13/f2c 0 2024-03-11T19:55:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:0/474: rmdir de/d11/d23/d32/d34 39 2024-03-11T19:55:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:0/475: readlink de/d11/d23/d32/d34/d39/l5d 0 2024-03-11T19:55:57.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/429: rename d1/d21/f2e to d1/d21/d35/d67/f8c 0 2024-03-11T19:55:57.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/478: write d6/d7e/d81/d27/f47 [4630040,53200] 0 2024-03-11T19:55:57.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/479: write d6/d5b/f78 [536463,37699] 0 2024-03-11T19:55:57.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/480: readlink d6/d7e/d81/d27/d2c/l2d 0 2024-03-11T19:55:57.481 INFO:tasks.workunit.client.0.smithi107.stdout:1/481: creat d6/d7e/d81/f98 x:0 0 0 2024-03-11T19:55:57.481 INFO:tasks.workunit.client.0.smithi107.stdout:1/482: chown d6/d7e/d81/d27/d2c/d32/f7f 889775 1 2024-03-11T19:55:57.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/453: write d4/d7/d1f/d2b/d19/f23 [132541,59564] 0 2024-03-11T19:55:57.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/454: symlink d4/d7/d1f/d2b/l89 0 2024-03-11T19:55:57.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/455: chown d4/d7/d2f/d33/d3a/d79 451747 1 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/430: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/431: creat d1/d21/d35/f8d x:0 0 0 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/456: creat d4/d7/db/d5c/f8a x:0 0 0 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/457: readlink d4/d7/db/lf 0 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/458: write d4/d7/d2f/d3c/f81 [669265,95447] 0 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/459: read - d4/d7/d1f/d82/f83 zero size 2024-03-11T19:55:57.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/460: write d4/d7/d1f/d2b/d19/f70 [1051655,51655] 0 2024-03-11T19:55:57.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/432: mkdir d1/d1a/d8e 0 2024-03-11T19:55:57.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/433: symlink d1/d38/d75/d76/l8f 0 2024-03-11T19:55:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/436: dwrite d1/d10/d13/d6c/f44 [4194304,4194304] 0 2024-03-11T19:55:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/437: write d1/d10/d21/d43/d48/f50 [811071,94546] 0 2024-03-11T19:55:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/438: rmdir d1/d10/d21 39 2024-03-11T19:55:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/439: rename d1/d10/d13/d6c/d2d/f6d to d1/d45/f86 0 2024-03-11T19:55:57.515 INFO:tasks.workunit.client.0.smithi107.stdout:1/483: dwrite d6/d28/d60/f63 [0,4194304] 0 2024-03-11T19:55:57.515 INFO:tasks.workunit.client.0.smithi107.stdout:1/484: write d6/d7e/d81/d27/d2c/d32/f8d [658154,112724] 0 2024-03-11T19:55:57.517 INFO:tasks.workunit.client.0.smithi107.stdout:3/440: rmdir d1/d10/d4e/d73/d6b 0 2024-03-11T19:55:57.518 INFO:tasks.workunit.client.0.smithi107.stdout:1/485: symlink d6/d28/l99 0 2024-03-11T19:55:57.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/486: rmdir d6/d7e/d81/d27/d2c/d32/d58 39 2024-03-11T19:55:57.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/441: mknod d1/d10/d13/d4a/c87 0 2024-03-11T19:55:57.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/442: fdatasync d1/d10/d21/d43/d4f/f59 0 2024-03-11T19:55:57.521 INFO:tasks.workunit.client.0.smithi107.stdout:1/487: link d6/d7e/d81/d27/f3e d6/d19/d3d/d82/f9a 0 2024-03-11T19:55:57.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/443: mkdir d1/d10/d21/d43/d48/d88 0 2024-03-11T19:55:57.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/444: chown d1/lc 35907 1 2024-03-11T19:55:57.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/445: rmdir d1/d10/d21/d43/d48/d88 0 2024-03-11T19:55:57.529 INFO:tasks.workunit.client.0.smithi107.stdout:2/434: dwrite d1/d38/d75/f7d [0,4194304] 0 2024-03-11T19:55:57.529 INFO:tasks.workunit.client.0.smithi107.stdout:2/435: fsync d1/d21/d35/f5c 0 2024-03-11T19:55:57.532 INFO:tasks.workunit.client.0.smithi107.stdout:4/461: dwrite d4/d7/d1f/d2b/d24/d26/f51 [0,4194304] 0 2024-03-11T19:55:57.532 INFO:tasks.workunit.client.0.smithi107.stdout:4/462: stat d4/d7/d1f/d2b/d19/f2a 0 2024-03-11T19:55:57.532 INFO:tasks.workunit.client.0.smithi107.stdout:4/463: readlink d4/d7/d1f/l40 0 2024-03-11T19:55:57.543 INFO:tasks.workunit.client.0.smithi107.stdout:4/464: symlink d4/d7/db/d5e/l8b 0 2024-03-11T19:55:57.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/465: creat d4/d7/d1f/d2b/d24/d26/d29/d36/f8c x:0 0 0 2024-03-11T19:55:57.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/466: fdatasync d4/d7/d1f/d2b/d24/f5b 0 2024-03-11T19:55:57.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/392: sync 2024-03-11T19:55:57.547 INFO:tasks.workunit.client.0.smithi107.stdout:9/446: sync 2024-03-11T19:55:57.547 INFO:tasks.workunit.client.0.smithi107.stdout:4/467: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/c8d 0 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/393: stat dd/de/d49/f73 0 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:7/495: sync 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:7/496: chown d6/da/c33 136 1 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:7/497: readlink d6/da/d16/d5c/d7d/d1b/d76/l83 0 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/394: write dd/f4c [5183985,115473] 0 2024-03-11T19:55:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/395: fdatasync dd/d13/d18/f5c 0 2024-03-11T19:55:57.554 INFO:tasks.workunit.client.0.smithi107.stdout:4/468: mkdir d4/d7/d1f/d2b/d19/d75/d8e 0 2024-03-11T19:55:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:9/447: write d4/d1d/d1f/d28/d2c/d96/f7a [3809500,36490] 0 2024-03-11T19:55:57.579 INFO:tasks.workunit.client.0.smithi107.stdout:9/448: mknod d4/d1d/d1f/cab 0 2024-03-11T19:55:57.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/436: dwrite d1/f10 [0,4194304] 0 2024-03-11T19:55:57.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/498: dwrite d6/da/f7c [0,4194304] 0 2024-03-11T19:55:57.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/446: dwrite d1/d10/d21/d43/d48/f56 [0,4194304] 0 2024-03-11T19:55:57.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/447: readlink d1/d10/d26/l4b 0 2024-03-11T19:55:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/448: symlink d1/d10/d26/l89 0 2024-03-11T19:55:57.604 INFO:tasks.workunit.client.0.smithi107.stdout:3/449: mknod d1/d10/d2a/c8a 0 2024-03-11T19:55:57.604 INFO:tasks.workunit.client.0.smithi107.stdout:3/450: readlink d1/l8 0 2024-03-11T19:55:57.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/488: dwrite d6/f55 [0,4194304] 0 2024-03-11T19:55:57.615 INFO:tasks.workunit.client.0.smithi107.stdout:4/469: dwrite d4/d7/d2f/f68 [0,4194304] 0 2024-03-11T19:55:57.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/470: link d4/d7/db/c45 d4/d7/db/c8f 0 2024-03-11T19:55:57.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/471: rename d4/d7/d1f/d2b/d19/l1c to d4/d7/d1f/d2b/d24/d26/d29/d36/l90 0 2024-03-11T19:55:57.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/396: dwrite dd/d16/f1e [0,4194304] 0 2024-03-11T19:55:57.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/397: truncate dd/f2e 333602 0 2024-03-11T19:55:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/398: stat dd/de/f3b 0 2024-03-11T19:55:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/399: getdents dd/de 0 2024-03-11T19:55:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/400: write dd/f2e [551703,8584] 0 2024-03-11T19:55:57.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/472: rmdir d4/d7/d1f/d2b/d24/d26 39 2024-03-11T19:55:57.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/473: creat d4/d7/d2f/d3c/f91 x:0 0 0 2024-03-11T19:55:57.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/437: dwrite d1/d1a/d1e/d26/f62 [4194304,4194304] 0 2024-03-11T19:55:57.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/438: fsync d1/f2b 0 2024-03-11T19:55:57.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/449: dwrite d4/d1d/d4b/f81 [0,4194304] 0 2024-03-11T19:55:57.642 INFO:tasks.workunit.client.0.smithi107.stdout:2/439: symlink d1/d1a/d1e/l90 0 2024-03-11T19:55:57.644 INFO:tasks.workunit.client.0.smithi107.stdout:2/440: symlink d1/d38/l91 0 2024-03-11T19:55:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:2/441: chown d1/l4e 342355103 1 2024-03-11T19:55:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/489: dwrite d6/d7e/d81/f21 [0,4194304] 0 2024-03-11T19:55:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/490: chown d6/d19/d3d/d62 41298846 1 2024-03-11T19:55:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/491: creat d6/d28/d60/f9b x:0 0 0 2024-03-11T19:55:57.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/442: mkdir d1/d38/d75/d92 0 2024-03-11T19:55:57.647 INFO:tasks.workunit.client.0.smithi107.stdout:2/443: symlink d1/d1a/d34/d4f/l93 0 2024-03-11T19:55:57.650 INFO:tasks.workunit.client.0.smithi107.stdout:4/474: read d4/d7/d2f/d3b/f4e [3807625,50453] 0 2024-03-11T19:55:57.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/451: dwrite d1/d10/d13/d6c/f32 [0,4194304] 0 2024-03-11T19:55:57.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/452: mkdir d1/d10/d2a/d7e/d8b 0 2024-03-11T19:55:57.656 INFO:tasks.workunit.client.0.smithi107.stdout:3/453: truncate d1/d10/d4e/d73/f20 3404545 0 2024-03-11T19:55:57.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/401: dwrite dd/de/d1b/d20/f43 [0,4194304] 0 2024-03-11T19:55:57.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/402: dread dd/d16/f33 [0,4194304] 0 2024-03-11T19:55:57.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/492: dread d6/d28/d60/f96 [0,4194304] 0 2024-03-11T19:55:57.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/493: chown d6/d7e/d81/d27/l3b 25556 1 2024-03-11T19:55:57.687 INFO:tasks.workunit.client.0.smithi107.stdout:1/494: fsync d6/f10 0 2024-03-11T19:55:57.687 INFO:tasks.workunit.client.0.smithi107.stdout:1/495: stat d6/d7e/d81/d27/c56 0 2024-03-11T19:55:57.689 INFO:tasks.workunit.client.0.smithi107.stdout:1/496: symlink d6/d7e/d81/l9c 0 2024-03-11T19:55:57.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/497: getdents d6/d19/d3d 0 2024-03-11T19:55:57.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/498: truncate d6/d28/f7c 1087092 0 2024-03-11T19:55:57.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/454: dread d1/d10/d13/d4a/d4c/f2f [0,4194304] 0 2024-03-11T19:55:57.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/499: symlink d6/d7e/d88/l9d 0 2024-03-11T19:55:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:4/475: dwrite d4/d7/d2f/d3b/f4e [0,4194304] 0 2024-03-11T19:55:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:3/455: mkdir d1/d10/d4e/d8c 0 2024-03-11T19:55:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:3/456: write d1/d10/d13/d6c/d5d/f70 [695090,113193] 0 2024-03-11T19:55:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:1/500: symlink d6/d28/d60/d95/l9e 0 2024-03-11T19:55:57.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/476: mkdir d4/d7/d1f/d2b/d24/d92 0 2024-03-11T19:55:57.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/477: fdatasync d4/d7/d1f/d2b/d24/d26/d29/f30 0 2024-03-11T19:55:57.695 INFO:tasks.workunit.client.0.smithi107.stdout:8/403: dread dd/f4c [0,4194304] 0 2024-03-11T19:55:57.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/501: symlink d6/d28/d60/d95/l9f 0 2024-03-11T19:55:57.696 INFO:tasks.workunit.client.0.smithi107.stdout:2/444: dwrite d1/f10 [0,4194304] 0 2024-03-11T19:55:57.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/457: mkdir d1/d10/d4e/d73/d8d 0 2024-03-11T19:55:57.696 INFO:tasks.workunit.client.0.smithi107.stdout:1/502: stat d6/d7e/d81/l2a 0 2024-03-11T19:55:57.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/445: rename d1/l4e to d1/d1d/d80/l94 0 2024-03-11T19:55:57.700 INFO:tasks.workunit.client.0.smithi107.stdout:3/458: rename d1/d10/d13/d6c/d5d/f70 to d1/d10/d21/d43/d48/f8e 0 2024-03-11T19:55:57.742 INFO:tasks.workunit.client.0.smithi107.stdout:1/503: dwrite d6/d28/f4b [4194304,4194304] 0 2024-03-11T19:55:57.742 INFO:tasks.workunit.client.0.smithi107.stdout:8/404: dwrite dd/de/d49/f44 [0,4194304] 0 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/476: sync 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/540: sync 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/403: sync 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/477: write de/d11/f2c [5211153,90648] 0 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/478: write de/d7a/f85 [973019,7567] 0 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/479: truncate de/d11/d74/f79 905238 0 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/541: readlink da/d31/d42/l20 0 2024-03-11T19:55:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/480: truncate de/d11/d23/d32/d34/f6c 251380 0 2024-03-11T19:55:57.747 INFO:tasks.workunit.client.0.smithi107.stdout:6/404: creat d10/d19/f89 x:0 0 0 2024-03-11T19:55:57.748 INFO:tasks.workunit.client.0.smithi107.stdout:0/481: truncate de/d11/d23/d26/f29 483246 0 2024-03-11T19:55:57.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/405: rmdir dd/d13/d32 39 2024-03-11T19:55:57.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/446: write d1/f10 [1688175,72410] 0 2024-03-11T19:55:57.751 INFO:tasks.workunit.client.0.smithi107.stdout:4/478: dwrite d4/d7/f2c [0,4194304] 0 2024-03-11T19:55:57.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/406: mknod dd/de/d1b/c77 0 2024-03-11T19:55:57.753 INFO:tasks.workunit.client.0.smithi107.stdout:2/447: rename d1/d1a/d1e/d26/f47 to d1/d38/d75/f95 0 2024-03-11T19:55:57.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/482: write de/d11/d23/d24/f69 [129147,113849] 0 2024-03-11T19:55:57.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/448: creat d1/d21/f96 x:0 0 0 2024-03-11T19:55:57.755 INFO:tasks.workunit.client.0.smithi107.stdout:0/483: truncate f3 7602318 0 2024-03-11T19:55:57.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/459: dwrite d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:55:57.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/460: fsync d1/d10/d2a/f38 0 2024-03-11T19:55:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/461: symlink d1/d10/d13/d6c/d2d/l8f 0 2024-03-11T19:55:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/462: mknod d1/d45/d85/c90 0 2024-03-11T19:55:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/463: write d1/d10/f6f [208407,66198] 0 2024-03-11T19:55:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/464: write d1/d10/d26/f58 [739581,50664] 0 2024-03-11T19:55:57.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/465: rmdir d1/d10/d13 39 2024-03-11T19:55:57.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/466: readlink d1/d10/d21/d43/l74 0 2024-03-11T19:55:57.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/467: truncate d1/d10/d26/f58 1222895 0 2024-03-11T19:55:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:3/468: mkdir d1/d10/d13/d6c/d2d/d4d/d91 0 2024-03-11T19:55:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/469: chown d1/d10/d2a 16935 1 2024-03-11T19:55:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/470: creat d1/d10/d21/d43/d4f/f92 x:0 0 0 2024-03-11T19:55:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:3/471: truncate d1/d10/d21/d43/d48/f69 3653248 0 2024-03-11T19:55:57.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/479: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:55:57.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/542: dread da/d31/d42/f28 [0,4194304] 0 2024-03-11T19:55:57.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/543: chown da/d14/d18/d6d 367 1 2024-03-11T19:55:57.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/480: mkdir d4/d7/d1f/d2b/d93 0 2024-03-11T19:55:57.779 INFO:tasks.workunit.client.0.smithi107.stdout:5/544: creat da/d14/d18/d6d/fa7 x:0 0 0 2024-03-11T19:55:57.781 INFO:tasks.workunit.client.0.smithi107.stdout:5/545: creat da/d14/d18/d44/da5/fa8 x:0 0 0 2024-03-11T19:55:57.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/407: write dd/de/d1b/d20/f29 [3553871,117543] 0 2024-03-11T19:55:57.799 INFO:tasks.workunit.client.0.smithi107.stdout:1/504: dwrite d6/d7e/d81/d27/d2c/f45 [4194304,4194304] 0 2024-03-11T19:55:57.801 INFO:tasks.workunit.client.0.smithi107.stdout:6/405: dwrite d10/d19/d3b/f64 [0,4194304] 0 2024-03-11T19:55:57.804 INFO:tasks.workunit.client.0.smithi107.stdout:0/484: write de/f16 [2244947,22634] 0 2024-03-11T19:55:57.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/505: getdents d6/d19 0 2024-03-11T19:55:57.814 INFO:tasks.workunit.client.0.smithi107.stdout:2/449: dwrite d1/d1a/d34/d4f/f83 [0,4194304] 0 2024-03-11T19:55:57.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/472: dwrite d1/d10/f57 [0,4194304] 0 2024-03-11T19:55:57.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/450: mkdir d1/d1d/d64/d97 0 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:1/506: dread d6/d7e/d81/d27/d2c/d32/f8d [0,4194304] 0 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:1/507: chown d6/d7e/d81/d27/f7a 9520 1 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:1/508: dread - d6/d7e/d81/f98 zero size 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/451: creat d1/d38/d75/d76/f98 x:0 0 0 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/452: write d1/d21/d35/f8d [638183,127805] 0 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/453: chown d1/d1a/f7c 0 1 2024-03-11T19:55:57.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/499: sync 2024-03-11T19:55:57.824 INFO:tasks.workunit.client.0.smithi107.stdout:9/450: sync 2024-03-11T19:55:57.824 INFO:tasks.workunit.client.0.smithi107.stdout:2/454: unlink d1/c13 0 2024-03-11T19:55:57.824 INFO:tasks.workunit.client.0.smithi107.stdout:2/455: fsync d1/d1d/d64/f8a 0 2024-03-11T19:55:57.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/456: rename d1/d21/d35/f78 to d1/d1d/d64/f99 0 2024-03-11T19:55:57.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/457: chown d1/d1a/d1e/l25 83170939 1 2024-03-11T19:55:57.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/406: dread d10/d19/d3b/f64 [0,4194304] 0 2024-03-11T19:55:57.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/485: dread de/f44 [0,4194304] 0 2024-03-11T19:55:57.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/486: rename de/l20 to de/d11/d74/l96 0 2024-03-11T19:55:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/500: dwrite d6/d21/f41 [4194304,4194304] 0 2024-03-11T19:55:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/501: readlink d6/da/l95 0 2024-03-11T19:55:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:7/502: rename d6/da/d4d/d77 to d6/da/d16/d5c/d7d/d60/d61/d9c 0 2024-03-11T19:55:57.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/487: read de/d11/d23/d32/d34/d39/d3a/f5e [1607147,1511] 0 2024-03-11T19:55:57.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/488: fsync de/d11/d23/d32/d34/d39/f53 0 2024-03-11T19:55:57.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/481: dwrite d4/fc [0,4194304] 0 2024-03-11T19:55:57.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/482: write d4/d7/d1f/f52 [930035,2704] 0 2024-03-11T19:55:57.880 INFO:tasks.workunit.client.0.smithi107.stdout:9/451: write d4/d1d/f3b [1436366,54850] 0 2024-03-11T19:55:57.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/407: dread d10/d19/d38/d60/f69 [0,4194304] 0 2024-03-11T19:55:57.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/408: mkdir d10/d19/d3b/d70/d8a 0 2024-03-11T19:55:57.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/409: write d10/d19/f61 [6650450,15302] 0 2024-03-11T19:55:57.881 INFO:tasks.workunit.client.0.smithi107.stdout:3/473: dwrite d1/d10/d4e/d73/f20 [0,4194304] 0 2024-03-11T19:55:57.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/410: creat d10/d19/d3b/f8b x:0 0 0 2024-03-11T19:55:57.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/411: read - d10/d19/d3b/f8b zero size 2024-03-11T19:55:57.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/412: chown d10/d16/d1f/f79 119 1 2024-03-11T19:55:57.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/413: write d10/d16/d24/f44 [9029116,110552] 0 2024-03-11T19:55:57.884 INFO:tasks.workunit.client.0.smithi107.stdout:3/474: truncate d1/d10/d13/d6c/f44 160949 0 2024-03-11T19:55:57.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/414: truncate d10/d19/d38/f5c 333216 0 2024-03-11T19:55:57.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/415: write d10/d16/d1f/d51/f6c [1719448,58063] 0 2024-03-11T19:55:57.886 INFO:tasks.workunit.client.0.smithi107.stdout:6/416: fsync d10/d19/f31 0 2024-03-11T19:55:57.888 INFO:tasks.workunit.client.0.smithi107.stdout:7/503: write d6/da/d16/f6f [596650,111582] 0 2024-03-11T19:55:57.889 INFO:tasks.workunit.client.0.smithi107.stdout:7/504: chown d6/d9/d4e 1714 1 2024-03-11T19:55:57.891 INFO:tasks.workunit.client.0.smithi107.stdout:8/408: dwrite dd/d16/f1e [0,4194304] 0 2024-03-11T19:55:57.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/417: creat d10/d19/d38/d40/f8c x:0 0 0 2024-03-11T19:55:57.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/475: rmdir d1 39 2024-03-11T19:55:57.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/505: rename d6/d9/l89 to d6/da/d16/d5c/d7d/d60/d61/d78/l9d 0 2024-03-11T19:55:57.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/489: dread de/f44 [0,4194304] 0 2024-03-11T19:55:57.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/509: dwrite d6/d7e/d81/d27/d2c/d72/f8a [0,4194304] 0 2024-03-11T19:55:57.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/483: read d4/d7/d1f/d2b/d24/f3e [2534799,35814] 0 2024-03-11T19:55:57.929 INFO:tasks.workunit.client.0.smithi107.stdout:4/484: dread d4/d7/d1f/d2b/d24/d26/d29/d36/f7a [0,4194304] 0 2024-03-11T19:55:57.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/476: write d1/d10/d13/f2c [1304117,115749] 0 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:4/485: unlink d4/d7/d2f/d33/c54 0 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:4/486: readlink d4/d7/d2f/d33/d3a/l61 0 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:3/477: rmdir d1/d10/d21/d43/d4f 39 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:4/487: chown d4/d7/d2f/d33/f88 3615057 1 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/452: dwrite d4/d1d/d5c/fa6 [0,4194304] 0 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/453: dread - d4/d1d/d1f/d28/d2c/d56/f94 zero size 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/454: write d4/d1d/f3b [387251,20655] 0 2024-03-11T19:55:57.934 INFO:tasks.workunit.client.0.smithi107.stdout:3/478: creat d1/d10/d13/d4a/f93 x:0 0 0 2024-03-11T19:55:57.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/479: chown d1/f6e 1063871 1 2024-03-11T19:55:57.936 INFO:tasks.workunit.client.0.smithi107.stdout:9/455: truncate d4/d1d/d4b/f81 212834 0 2024-03-11T19:55:57.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/480: creat d1/d10/d13/d6c/d5d/f94 x:0 0 0 2024-03-11T19:55:57.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/481: readlink d1/l8 0 2024-03-11T19:55:57.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/482: chown d1/d10/d26/l4b 81869 1 2024-03-11T19:55:57.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/483: creat d1/d10/d4e/d73/d8d/f95 x:0 0 0 2024-03-11T19:55:57.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/484: chown d1/d10/d13/d4a/f93 287 1 2024-03-11T19:55:57.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/485: dread - d1/d10/d21/d43/f84 zero size 2024-03-11T19:55:57.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/456: truncate d4/d1d/f3b 4049315 0 2024-03-11T19:55:57.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/457: write d4/d1c/f7e [787313,129] 0 2024-03-11T19:55:57.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/458: truncate d4/d1d/d1f/d28/f67 355106 0 2024-03-11T19:55:57.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/459: mknod d4/d1d/d5c/cac 0 2024-03-11T19:55:57.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/460: getdents d4/d1c/d47/d8a 0 2024-03-11T19:55:57.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/490: dwrite de/d11/d74/f79 [0,4194304] 0 2024-03-11T19:55:57.958 INFO:tasks.workunit.client.0.smithi107.stdout:7/506: dwrite d6/da/d16/d5c/f73 [0,4194304] 0 2024-03-11T19:55:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/409: dread dd/f21 [0,4194304] 0 2024-03-11T19:55:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/491: dread de/f59 [0,4194304] 0 2024-03-11T19:55:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:7/507: getdents d6/d9/d57/d65 0 2024-03-11T19:55:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/410: link dd/d13/d18/f37 dd/d13/d32/f78 0 2024-03-11T19:55:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:0/492: creat de/d7a/f97 x:0 0 0 2024-03-11T19:55:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:0/493: fsync de/d11/d23/d24/f60 0 2024-03-11T19:55:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/494: fsync de/d11/d23/d32/d34/f6c 0 2024-03-11T19:55:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/495: chown de/d11/d23/d32/d34/d39/d55/l5c 29 1 2024-03-11T19:55:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/496: write de/f88 [704827,111796] 0 2024-03-11T19:55:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/497: truncate de/f67 615410 0 2024-03-11T19:55:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:7/508: creat d6/d9/d57/f9e x:0 0 0 2024-03-11T19:55:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/411: mkdir dd/de/d1b/d20/d79 0 2024-03-11T19:55:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/412: fdatasync dd/de/d1b/d20/d5e/f65 0 2024-03-11T19:55:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/413: chown l9 3396 1 2024-03-11T19:55:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/414: write dd/de/d49/f5b [853061,18925] 0 2024-03-11T19:55:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/418: dwrite d10/d16/d1f/f79 [0,4194304] 0 2024-03-11T19:55:57.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/509: dread d6/f17 [0,4194304] 0 2024-03-11T19:55:57.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/510: getdents d6/da/d16/d5c/d7d/d60/d61/d9c 0 2024-03-11T19:55:57.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/511: symlink d6/d9/d57/l9f 0 2024-03-11T19:55:57.997 INFO:tasks.workunit.client.0.smithi107.stdout:7/512: rmdir d6/da/d16 39 2024-03-11T19:55:58.001 INFO:tasks.workunit.client.0.smithi107.stdout:9/461: dwrite d4/d1d/d1f/d28/d2c/fa5 [0,4194304] 0 2024-03-11T19:55:58.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/462: mknod d4/d1d/d1f/d28/d5d/cad 0 2024-03-11T19:55:58.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/463: write d4/d1d/d4b/f59 [348715,37944] 0 2024-03-11T19:55:58.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/464: rename d4/d1d/d1f/d28/d2c/d56 to d4/d9/dae 0 2024-03-11T19:55:58.011 INFO:tasks.workunit.client.0.smithi107.stdout:3/486: dread d1/f5 [4194304,4194304] 0 2024-03-11T19:55:58.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/488: dwrite d4/d7/d1f/d2b/d19/f2a [0,4194304] 0 2024-03-11T19:55:58.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/489: fsync d4/d7/d2f/d33/f88 0 2024-03-11T19:55:58.024 INFO:tasks.workunit.client.0.smithi107.stdout:6/419: dwrite d10/d19/f61 [0,4194304] 0 2024-03-11T19:55:58.024 INFO:tasks.workunit.client.0.smithi107.stdout:5/546: sync 2024-03-11T19:55:58.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/465: dread d4/d9/dae/d6d/f2d [0,4194304] 0 2024-03-11T19:55:58.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/420: mkdir d10/d16/d2b/d8d 0 2024-03-11T19:55:58.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/547: symlink da/d31/d42/la9 0 2024-03-11T19:55:58.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/548: readlink da/d14/l5c 0 2024-03-11T19:55:58.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/549: chown da/d14/d18/d1e/c37 19500185 1 2024-03-11T19:55:58.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/550: creat da/d14/d18/d1e/d9b/d73/faa x:0 0 0 2024-03-11T19:55:58.034 INFO:tasks.workunit.client.0.smithi107.stdout:9/466: mknod d4/d1c/d47/d8a/caf 0 2024-03-11T19:55:58.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/490: rename d4/d7/c53 to d4/d7/d2f/d33/d3a/d79/c94 0 2024-03-11T19:55:58.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/467: mkdir d4/d9/dae/d6d/db0 0 2024-03-11T19:55:58.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/468: chown d4/d1d/d5c/d64 508368468 1 2024-03-11T19:55:58.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/491: unlink d4/d7/d1f/d2b/d24/d26/f38 0 2024-03-11T19:55:58.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/492: read - d4/d7/d1f/d82/f83 zero size 2024-03-11T19:55:58.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/493: rmdir d4/d7/db/d5e 39 2024-03-11T19:55:58.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/415: dwrite dd/d13/f6c [0,4194304] 0 2024-03-11T19:55:58.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/494: symlink d4/l95 0 2024-03-11T19:55:58.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/495: creat d4/d7/d2f/d3c/f96 x:0 0 0 2024-03-11T19:55:58.055 INFO:tasks.workunit.client.0.smithi107.stdout:4/496: mknod d4/d7/d1f/d2b/d19/d75/c97 0 2024-03-11T19:55:58.057 INFO:tasks.workunit.client.0.smithi107.stdout:4/497: creat d4/d7/f98 x:0 0 0 2024-03-11T19:55:58.071 INFO:tasks.workunit.client.0.smithi107.stdout:3/487: dwrite d1/d45/f75 [0,4194304] 0 2024-03-11T19:55:58.071 INFO:tasks.workunit.client.0.smithi107.stdout:7/513: dwrite d6/da/d16/d5c/f70 [0,4194304] 0 2024-03-11T19:55:58.071 INFO:tasks.workunit.client.0.smithi107.stdout:4/498: dread d4/d7/d1f/d2b/d24/f5b [0,4194304] 0 2024-03-11T19:55:58.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/469: dwrite d4/d9/dae/d6d/f2d [0,4194304] 0 2024-03-11T19:55:58.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/470: chown d4/f58 132581 1 2024-03-11T19:55:58.096 INFO:tasks.workunit.client.0.smithi107.stdout:9/471: mknod d4/d1d/d1f/d78/cb1 0 2024-03-11T19:55:58.097 INFO:tasks.workunit.client.0.smithi107.stdout:9/472: fsync d4/d1d/d1f/d28/d2c/f62 0 2024-03-11T19:55:58.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/473: write d4/d1d/d1f/d28/fa2 [793924,16899] 0 2024-03-11T19:55:58.099 INFO:tasks.workunit.client.0.smithi107.stdout:5/551: dwrite da/d14/d38/d46/f61 [4194304,4194304] 0 2024-03-11T19:55:58.101 INFO:tasks.workunit.client.0.smithi107.stdout:6/421: dwrite d10/d19/f61 [4194304,4194304] 0 2024-03-11T19:55:58.101 INFO:tasks.workunit.client.0.smithi107.stdout:9/474: link d4/d1d/d1f/d28/d2c/f34 d4/d1d/d1f/d78/fb2 0 2024-03-11T19:55:58.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/552: rmdir da/d14/d38 39 2024-03-11T19:55:58.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/553: write da/d31/d42/f1f [4741851,89558] 0 2024-03-11T19:55:58.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/554: dread - da/d14/d38/f78 zero size 2024-03-11T19:55:58.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/416: dwrite dd/de/d49/f42 [0,4194304] 0 2024-03-11T19:55:58.105 INFO:tasks.workunit.client.0.smithi107.stdout:6/422: getdents d10/d3d 0 2024-03-11T19:55:58.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/417: getdents dd/de/d1b/d20 0 2024-03-11T19:55:58.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/418: readlink dd/d13/d32/d51/l5d 0 2024-03-11T19:55:58.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/419: chown dd/d13/d18/f75 262697702 1 2024-03-11T19:55:58.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/420: chown dd/f58 42694914 1 2024-03-11T19:55:58.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/555: rename da/d14/d18/d44/d8b to da/d14/d43/dab 0 2024-03-11T19:55:58.115 INFO:tasks.workunit.client.0.smithi107.stdout:4/499: dwrite d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:55:58.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/458: sync 2024-03-11T19:55:58.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/459: dread - d1/d1a/d1e/d26/f7a zero size 2024-03-11T19:55:58.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/460: readlink d1/l1c 0 2024-03-11T19:55:58.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/461: creat d1/d38/d75/f9a x:0 0 0 2024-03-11T19:55:58.119 INFO:tasks.workunit.client.0.smithi107.stdout:7/514: dwrite d6/da/d16/d5c/f54 [0,4194304] 0 2024-03-11T19:55:58.125 INFO:tasks.workunit.client.0.smithi107.stdout:3/488: dwrite d1/d45/f63 [4194304,4194304] 0 2024-03-11T19:55:58.125 INFO:tasks.workunit.client.0.smithi107.stdout:2/462: truncate d1/d1a/d34/d4f/d63/f6f 1120970 0 2024-03-11T19:55:58.125 INFO:tasks.workunit.client.0.smithi107.stdout:4/500: rename d4/d7/f1d to d4/d7/d2f/d33/d7b/f99 0 2024-03-11T19:55:58.125 INFO:tasks.workunit.client.0.smithi107.stdout:4/501: chown d4/d7/d1f/f52 115367748 1 2024-03-11T19:55:58.126 INFO:tasks.workunit.client.0.smithi107.stdout:7/515: creat d6/da/d49/fa0 x:0 0 0 2024-03-11T19:55:58.130 INFO:tasks.workunit.client.0.smithi107.stdout:2/463: creat d1/d1a/d1e/f9b x:0 0 0 2024-03-11T19:55:58.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/502: mkdir d4/d7/d1f/d2b/d19/d9a 0 2024-03-11T19:55:58.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/503: mknod d4/d7/d1f/d2b/d24/d92/c9b 0 2024-03-11T19:55:58.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/510: sync 2024-03-11T19:55:58.139 INFO:tasks.workunit.client.0.smithi107.stdout:0/498: sync 2024-03-11T19:55:58.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/421: dread dd/de/d1b/f1d [0,4194304] 0 2024-03-11T19:55:58.148 INFO:tasks.workunit.client.0.smithi107.stdout:9/475: dwrite d4/d1d/d1f/d28/d2c/d96/fa4 [0,4194304] 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/511: mknod d6/d7e/d81/d27/d2c/ca0 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/489: write d1/d45/f86 [2409505,49632] 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/556: write da/d14/d38/d59/d5f/f6a [350092,50311] 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/557: chown da/d31/d42 2058806 1 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/490: dread - d1/d10/d13/d4a/d4c/d49/f54 zero size 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/491: fdatasync d1/d10/d21/d43/f84 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/558: write f4 [9355995,21856] 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/492: fsync d1/f5 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/493: fsync d1/d10/f57 0 2024-03-11T19:55:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/559: write da/d14/d18/d1e/d9b/d73/f92 [884415,22768] 0 2024-03-11T19:55:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/512: symlink d6/d7e/d88/la1 0 2024-03-11T19:55:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/513: dread - d6/f68 zero size 2024-03-11T19:55:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/494: rename d1/d10/d26/l4b to d1/d10/d4e/d73/l96 0 2024-03-11T19:55:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:1/514: mkdir d6/d19/d3d/da2 0 2024-03-11T19:55:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:1/515: readlink d6/d7e/d81/l1f 0 2024-03-11T19:55:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/495: unlink d1/d10/d13/c1d 0 2024-03-11T19:55:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/496: readlink d1/d10/d13/d6c/d5d/l77 0 2024-03-11T19:55:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:5/560: fdatasync da/d14/d38/d59/d5f/f6a 0 2024-03-11T19:55:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:5/561: chown da/d14/d18/d1e/c34 2 1 2024-03-11T19:55:58.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/516: mkdir d6/d19/d3d/d62/da3 0 2024-03-11T19:55:58.164 INFO:tasks.workunit.client.0.smithi107.stdout:7/516: dwrite d6/d21/f41 [4194304,4194304] 0 2024-03-11T19:55:58.164 INFO:tasks.workunit.client.0.smithi107.stdout:7/517: creat d6/d9/d57/fa1 x:0 0 0 2024-03-11T19:55:58.165 INFO:tasks.workunit.client.0.smithi107.stdout:5/562: rename da/d14/d18/d1e/d9b/f97 to da/d14/d18/d1e/d9b/d72/fac 0 2024-03-11T19:55:58.166 INFO:tasks.workunit.client.0.smithi107.stdout:5/563: chown da/f86 20489 1 2024-03-11T19:55:58.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/564: mkdir da/d14/d43/d9e/dad 0 2024-03-11T19:55:58.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/565: creat da/d14/d18/d1e/d9b/d73/fae x:0 0 0 2024-03-11T19:55:58.191 INFO:tasks.workunit.client.0.smithi107.stdout:4/504: dread d4/d7/f2c [0,4194304] 0 2024-03-11T19:55:58.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/505: symlink d4/d7/d1f/d2b/d19/d75/d8e/l9c 0 2024-03-11T19:55:58.194 INFO:tasks.workunit.client.0.smithi107.stdout:4/506: mknod d4/d7/d1f/d2b/d24/d92/c9d 0 2024-03-11T19:55:58.194 INFO:tasks.workunit.client.0.smithi107.stdout:4/507: chown d4/d7/d2f/d33/l7f 37949124 1 2024-03-11T19:55:58.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/508: write d4/d7/d1f/d2b/d24/d26/d29/d36/f78 [434806,31076] 0 2024-03-11T19:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout:9/476: dwrite d4/f54 [0,4194304] 0 2024-03-11T19:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/422: dwrite dd/de/d1b/d20/f29 [4194304,4194304] 0 2024-03-11T19:55:58.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/464: dwrite d1/d1a/d34/d4f/d63/f66 [0,4194304] 0 2024-03-11T19:55:58.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/499: dread de/f3d [0,4194304] 0 2024-03-11T19:55:58.215 INFO:tasks.workunit.client.0.smithi107.stdout:1/517: dwrite d6/f15 [0,4194304] 0 2024-03-11T19:55:58.218 INFO:tasks.workunit.client.0.smithi107.stdout:9/477: dread d4/d1d/d5c/d64/f84 [0,4194304] 0 2024-03-11T19:55:58.218 INFO:tasks.workunit.client.0.smithi107.stdout:7/518: dread d6/d21/f3a [0,4194304] 0 2024-03-11T19:55:58.218 INFO:tasks.workunit.client.0.smithi107.stdout:9/478: stat d4/d1c/d47/d8a/ca1 0 2024-03-11T19:55:58.218 INFO:tasks.workunit.client.0.smithi107.stdout:3/497: dwrite d1/d10/d4e/d73/d8d/f95 [0,4194304] 0 2024-03-11T19:55:58.220 INFO:tasks.workunit.client.0.smithi107.stdout:7/519: mkdir d6/d9/d57/da2 0 2024-03-11T19:55:58.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/498: stat d1/d10/d13/d4a/d4c/d49/c83 0 2024-03-11T19:55:58.221 INFO:tasks.workunit.client.0.smithi107.stdout:0/500: creat de/d11/d23/d32/d34/d39/d3a/d72/f98 x:0 0 0 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/520: rename d6/da/f7c to d6/da/fa3 0 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/521: fdatasync d6/da/d16/d5c/f70 0 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/522: chown d6 3275494 1 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/523: chown d6/da/f1f 11 1 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/501: rmdir de/d11/d23/d26 39 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/502: readlink de/d11/l19 0 2024-03-11T19:55:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/503: dread - de/d11/d23/d24/f60 zero size 2024-03-11T19:55:58.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/518: unlink d6/f55 0 2024-03-11T19:55:58.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/519: write d6/f1b [554876,123624] 0 2024-03-11T19:55:58.228 INFO:tasks.workunit.client.0.smithi107.stdout:3/499: creat d1/d10/d2a/d7e/d8b/f97 x:0 0 0 2024-03-11T19:55:58.229 INFO:tasks.workunit.client.0.smithi107.stdout:0/504: mkdir de/d11/d23/d32/d34/d39/d3a/d72/d99 0 2024-03-11T19:55:58.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/505: symlink de/d11/d23/d26/l9a 0 2024-03-11T19:55:58.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/520: mkdir d6/d7e/da4 0 2024-03-11T19:55:58.232 INFO:tasks.workunit.client.0.smithi107.stdout:0/506: creat de/d11/d23/d32/d34/f9b x:0 0 0 2024-03-11T19:55:58.232 INFO:tasks.workunit.client.0.smithi107.stdout:0/507: chown de/d11/d23/d32/d34/d39/l52 252907 1 2024-03-11T19:55:58.232 INFO:tasks.workunit.client.0.smithi107.stdout:1/521: mkdir d6/d7e/da5 0 2024-03-11T19:55:58.235 INFO:tasks.workunit.client.0.smithi107.stdout:1/522: link d6/d7e/d81/f29 d6/d7e/d81/d27/d2c/d72/fa6 0 2024-03-11T19:55:58.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/566: dwrite da/d14/d38/d46/f9f [0,4194304] 0 2024-03-11T19:55:58.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/524: read d6/d9/f6b [4137185,80718] 0 2024-03-11T19:55:58.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/525: fsync d6/d9/d57/f5a 0 2024-03-11T19:55:58.251 INFO:tasks.workunit.client.0.smithi107.stdout:7/526: unlink d6/da/d16/d5c/d7d/d60/f72 0 2024-03-11T19:55:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:7/527: link d6/da/d16/f36 d6/d9/d4e/fa4 0 2024-03-11T19:55:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:7/528: chown d6/d9/f6b 1 1 2024-03-11T19:55:58.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/508: rename de/c30 to de/d11/d23/d32/d34/d39/d55/d7c/c9c 0 2024-03-11T19:55:58.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/465: dwrite d1/d1a/d34/d6a/f82 [0,4194304] 0 2024-03-11T19:55:58.265 INFO:tasks.workunit.client.0.smithi107.stdout:7/529: rename d6/d9/d57/d65/f6a to d6/da/d16/fa5 0 2024-03-11T19:55:58.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/509: creat de/d11/d23/f9d x:0 0 0 2024-03-11T19:55:58.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/466: symlink d1/d38/d75/l9c 0 2024-03-11T19:55:58.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/423: dwrite dd/de/d1b/d2c/f2d [0,4194304] 0 2024-03-11T19:55:58.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/424: chown dd/de/d1b/d2c/l30 8 1 2024-03-11T19:55:58.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/425: stat dd/f69 0 2024-03-11T19:55:58.272 INFO:tasks.workunit.client.0.smithi107.stdout:7/530: link d6/d21/f4c d6/d9/fa6 0 2024-03-11T19:55:58.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/510: mkdir de/d9e 0 2024-03-11T19:55:58.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/511: write de/d11/d23/d32/d34/f82 [986742,188] 0 2024-03-11T19:55:58.274 INFO:tasks.workunit.client.0.smithi107.stdout:0/512: read de/d11/d23/d24/f69 [99035,120830] 0 2024-03-11T19:55:58.274 INFO:tasks.workunit.client.0.smithi107.stdout:3/500: dread d1/d45/f63 [4194304,4194304] 0 2024-03-11T19:55:58.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/467: link d1/d1a/d1e/d41/l44 d1/d1a/d1e/l9d 0 2024-03-11T19:55:58.276 INFO:tasks.workunit.client.0.smithi107.stdout:7/531: rmdir d6/da/d16/d5c/d7d/d1b/d76 39 2024-03-11T19:55:58.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/468: read d1/d21/d35/d67/f8c [311956,73389] 0 2024-03-11T19:55:58.277 INFO:tasks.workunit.client.0.smithi107.stdout:8/426: creat dd/de/d1b/f7a x:0 0 0 2024-03-11T19:55:58.277 INFO:tasks.workunit.client.0.smithi107.stdout:8/427: chown dd/de/d1b/d20/c70 11744 1 2024-03-11T19:55:58.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/501: mkdir d1/d10/d21/d43/d48/d98 0 2024-03-11T19:55:58.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/502: readlink d1/l8 0 2024-03-11T19:55:58.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/469: link d1/c3b d1/d1a/d1e/d41/c9e 0 2024-03-11T19:55:58.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/428: rename f4 to dd/d13/d32/f7b 0 2024-03-11T19:55:58.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/513: dread de/d11/d23/d24/d4f/f78 [0,4194304] 0 2024-03-11T19:55:58.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/514: dread - de/d11/d23/d32/d34/d39/f6e zero size 2024-03-11T19:55:58.288 INFO:tasks.workunit.client.0.smithi107.stdout:7/532: dread d6/d9/f30 [0,4194304] 0 2024-03-11T19:55:58.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/523: dwrite d6/d7e/d81/d27/d2c/d72/f75 [4194304,4194304] 0 2024-03-11T19:55:58.291 INFO:tasks.workunit.client.0.smithi107.stdout:5/567: dwrite da/d14/d43/f9a [0,4194304] 0 2024-03-11T19:55:58.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/524: symlink d6/d7e/da4/la7 0 2024-03-11T19:55:58.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/525: fdatasync d6/f1d 0 2024-03-11T19:55:58.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/533: unlink d6/da/d16/d5c/d7d/d60/d61/l94 0 2024-03-11T19:55:58.305 INFO:tasks.workunit.client.0.smithi107.stdout:8/429: dread dd/de/d1b/d20/d5e/f65 [0,4194304] 0 2024-03-11T19:55:58.306 INFO:tasks.workunit.client.0.smithi107.stdout:5/568: creat da/d14/d38/d46/faf x:0 0 0 2024-03-11T19:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/569: rmdir da/d14/d96/d7c 0 2024-03-11T19:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/570: chown da/d14/d18/d1e/d9b/d73/fae 641 1 2024-03-11T19:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout:8/430: mknod dd/de/d1b/d20/d5e/c7c 0 2024-03-11T19:55:58.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/571: getdents da/d14/d18 0 2024-03-11T19:55:58.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/431: rmdir dd/de/d1b/d20/d79 0 2024-03-11T19:55:58.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/526: write d6/d28/f41 [384515,81067] 0 2024-03-11T19:55:58.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/534: write d6/d9/d4e/fa4 [382915,19317] 0 2024-03-11T19:55:58.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/527: symlink d6/d7e/da4/la8 0 2024-03-11T19:55:58.330 INFO:tasks.workunit.client.0.smithi107.stdout:2/470: dwrite d1/d1a/d34/d4f/f5d [0,4194304] 0 2024-03-11T19:55:58.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/503: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:58.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/504: chown d1/d10/d13/d6c/d2d/l33 56110984 1 2024-03-11T19:55:58.334 INFO:tasks.workunit.client.0.smithi107.stdout:2/471: mknod d1/d1a/d1e/c9f 0 2024-03-11T19:55:58.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/505: mkdir d1/d10/d13/d6c/d5d/d99 0 2024-03-11T19:55:58.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/506: write d1/d10/d21/d43/f84 [147025,27437] 0 2024-03-11T19:55:58.334 INFO:tasks.workunit.client.0.smithi107.stdout:2/472: creat d1/d38/d75/fa0 x:0 0 0 2024-03-11T19:55:58.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/473: creat d1/d1d/d64/fa1 x:0 0 0 2024-03-11T19:55:58.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/528: write d6/d7e/d81/d27/f3e [1341138,16533] 0 2024-03-11T19:55:58.346 INFO:tasks.workunit.client.0.smithi107.stdout:6/423: sync 2024-03-11T19:55:58.346 INFO:tasks.workunit.client.0.smithi107.stdout:6/424: readlink d10/d16/d1f/l7c 0 2024-03-11T19:55:58.346 INFO:tasks.workunit.client.0.smithi107.stdout:1/529: mknod d6/d7e/d81/d27/d2c/d7d/ca9 0 2024-03-11T19:55:58.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/474: symlink d1/d38/d75/la2 0 2024-03-11T19:55:58.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/475: truncate d1/d21/d35/f5c 653764 0 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/515: dwrite de/d11/d23/d26/f43 [0,4194304] 0 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/516: chown de/d58 12770325 1 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/517: stat de/d11/d23/d24/d4f/d89 0 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/530: fdatasync d6/d19/d3d/d82/f9a 0 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/531: dread - d6/d28/d60/f9b zero size 2024-03-11T19:55:58.349 INFO:tasks.workunit.client.0.smithi107.stdout:6/425: unlink d10/d19/d3b/l6f 0 2024-03-11T19:55:58.351 INFO:tasks.workunit.client.0.smithi107.stdout:1/532: symlink d6/d28/laa 0 2024-03-11T19:55:58.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/572: read da/d14/d18/d1e/f65 [1800325,46986] 0 2024-03-11T19:55:58.352 INFO:tasks.workunit.client.0.smithi107.stdout:6/426: mknod d10/d19/d38/d60/c8e 0 2024-03-11T19:55:58.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/573: getdents da/d14 0 2024-03-11T19:55:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/574: creat da/d14/d43/d9e/dad/fb0 x:0 0 0 2024-03-11T19:55:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:5/575: fsync da/f32 0 2024-03-11T19:55:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/432: dread dd/d13/f6c [0,4194304] 0 2024-03-11T19:55:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/433: stat dd/de/d1b/d2c 0 2024-03-11T19:55:58.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/535: dread d6/da/d16/d5c/f54 [0,4194304] 0 2024-03-11T19:55:58.373 INFO:tasks.workunit.client.0.smithi107.stdout:7/536: link d6/da/d49/c7a d6/d9/d57/d65/ca7 0 2024-03-11T19:55:58.376 INFO:tasks.workunit.client.0.smithi107.stdout:6/427: rename d10/d3d/l50 to d10/d16/d1f/d51/l8f 0 2024-03-11T19:55:58.376 INFO:tasks.workunit.client.0.smithi107.stdout:6/428: fdatasync d10/d19/d38/d60/f69 0 2024-03-11T19:55:58.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/537: rename d6/d9/f6b to d6/d9/d4e/fa8 0 2024-03-11T19:55:58.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/429: unlink d10/d16/f37 0 2024-03-11T19:55:58.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/533: dread d6/d7e/d81/d27/d2c/d72/f8a [0,4194304] 0 2024-03-11T19:55:58.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/534: rename d6/d7e/d81/l2a to d6/d7e/d81/d27/d2c/d72/lab 0 2024-03-11T19:55:58.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/434: dread dd/de/d49/f42 [0,4194304] 0 2024-03-11T19:55:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/476: dwrite d1/d21/d35/f8d [0,4194304] 0 2024-03-11T19:55:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/477: creat d1/d1a/fa3 x:0 0 0 2024-03-11T19:55:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/478: stat d1/d1a/d34/f42 0 2024-03-11T19:55:58.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/535: rename d6/d19/d3d to d6/d7e/da5/dac 0 2024-03-11T19:55:58.406 INFO:tasks.workunit.client.0.smithi107.stdout:2/479: mknod d1/d21/d35/d67/ca4 0 2024-03-11T19:55:58.406 INFO:tasks.workunit.client.0.smithi107.stdout:2/480: chown d1/d1a/d1e/l90 21 1 2024-03-11T19:55:58.406 INFO:tasks.workunit.client.0.smithi107.stdout:2/481: creat d1/d21/d35/fa5 x:0 0 0 2024-03-11T19:55:58.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/536: rename d6/d19/c5e to d6/d7e/d81/d27/d2c/d32/d58/cad 0 2024-03-11T19:55:58.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/537: creat d6/d28/d60/d95/fae x:0 0 0 2024-03-11T19:55:58.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/538: write d6/d7e/d81/d27/f7a [867363,58256] 0 2024-03-11T19:55:58.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/538: dwrite d6/da/f45 [0,4194304] 0 2024-03-11T19:55:58.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/539: symlink d6/d7e/da4/laf 0 2024-03-11T19:55:58.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/540: truncate d6/d28/f7c 215815 0 2024-03-11T19:55:58.429 INFO:tasks.workunit.client.0.smithi107.stdout:7/539: creat d6/da/d16/d5c/d7d/fa9 x:0 0 0 2024-03-11T19:55:58.430 INFO:tasks.workunit.client.0.smithi107.stdout:6/430: dwrite d10/d16/d1f/f25 [4194304,4194304] 0 2024-03-11T19:55:58.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/435: dwrite dd/d13/d18/f37 [0,4194304] 0 2024-03-11T19:55:58.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/436: stat dd/d13/d18 0 2024-03-11T19:55:58.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/540: mkdir d6/d21/daa 0 2024-03-11T19:55:58.432 INFO:tasks.workunit.client.0.smithi107.stdout:8/437: truncate dd/de/d1b/d20/f43 2348529 0 2024-03-11T19:55:58.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/438: mknod dd/c7d 0 2024-03-11T19:55:58.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/439: mkdir dd/de/d49/d7e 0 2024-03-11T19:55:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/431: rename d10/c12 to d10/d16/d24/c90 0 2024-03-11T19:55:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:6/432: creat d10/d16/d2b/f91 x:0 0 0 2024-03-11T19:55:58.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/433: fdatasync d10/d16/d24/f44 0 2024-03-11T19:55:58.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/509: sync 2024-03-11T19:55:58.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/510: truncate d4/d7/d2f/f65 1848776 0 2024-03-11T19:55:58.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/511: chown d4/d7/d1f/d2b/l89 249852 1 2024-03-11T19:55:58.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/541: dwrite d6/d7e/d81/f29 [0,4194304] 0 2024-03-11T19:55:58.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/542: read d6/d7e/d81/f24 [998396,80202] 0 2024-03-11T19:55:58.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/543: write d6/d7e/d81/d27/f7b [860101,68295] 0 2024-03-11T19:55:58.473 INFO:tasks.workunit.client.0.smithi107.stdout:8/440: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:55:58.475 INFO:tasks.workunit.client.0.smithi107.stdout:8/441: creat dd/d16/f7f x:0 0 0 2024-03-11T19:55:58.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/442: creat dd/de/d1b/d20/d5e/f80 x:0 0 0 2024-03-11T19:55:58.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/443: fsync dd/f69 0 2024-03-11T19:55:58.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/541: dread d6/da/f45 [0,4194304] 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/542: mkdir d6/d21/dab 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/543: read d6/d9/d4e/fa4 [3283601,7845] 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/544: stat d6/da/d16/d5c/d7d/l5d 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/544: write d6/d19/f43 [1759481,120617] 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/545: write d6/d9/f6d [5073336,9172] 0 2024-03-11T19:55:58.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/545: dread - d6/d28/f80 zero size 2024-03-11T19:55:58.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/546: stat d6/d7e/d88 0 2024-03-11T19:55:58.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/547: write d6/d7e/d81/d27/f7b [1899360,109774] 0 2024-03-11T19:55:58.505 INFO:tasks.workunit.client.0.smithi107.stdout:7/546: rmdir d6 39 2024-03-11T19:55:58.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/547: link d6/da/d16/d5c/c9a d6/da/d16/d91/d97/d9b/cac 0 2024-03-11T19:55:58.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/512: dwrite d4/d7/d1f/d2b/d24/d26/f84 [0,4194304] 0 2024-03-11T19:55:58.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/444: dread dd/de/d49/f73 [0,4194304] 0 2024-03-11T19:55:58.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/445: read - dd/de/d1b/f7a zero size 2024-03-11T19:55:58.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/513: link d4/d7/d2f/d33/l7f d4/d7/d1f/d2b/d19/d75/d8e/l9e 0 2024-03-11T19:55:58.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/446: mkdir dd/d13/d18/d81 0 2024-03-11T19:55:58.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/514: creat d4/d7/db/d5e/f9f x:0 0 0 2024-03-11T19:55:58.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/447: link dd/de/d1b/d20/f43 dd/de/d1b/d4b/f82 0 2024-03-11T19:55:58.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/448: chown dd/de/d24/d28/d47 0 1 2024-03-11T19:55:58.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/449: dread - dd/de/d1b/d20/f6a zero size 2024-03-11T19:55:58.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/515: getdents d4/d7/db/d5e 0 2024-03-11T19:55:58.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/434: dwrite d10/d16/d24/f68 [0,4194304] 0 2024-03-11T19:55:58.523 INFO:tasks.workunit.client.0.smithi107.stdout:4/516: mkdir d4/d7/da0 0 2024-03-11T19:55:58.523 INFO:tasks.workunit.client.0.smithi107.stdout:4/517: fdatasync d4/d7/d1f/f20 0 2024-03-11T19:55:58.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/548: dread d6/d28/f4b [0,4194304] 0 2024-03-11T19:55:58.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/548: read d6/da/d16/f2e [2861123,60553] 0 2024-03-11T19:55:58.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/549: write d6/da/d16/d5c/d7d/d1b/f1c [2691233,98793] 0 2024-03-11T19:55:58.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/550: write d6/da/d16/d5c/f54 [1344013,106166] 0 2024-03-11T19:55:58.533 INFO:tasks.workunit.client.0.smithi107.stdout:9/479: sync 2024-03-11T19:55:58.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/480: unlink d4/d9/f15 0 2024-03-11T19:55:58.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/481: mkdir d4/d1d/d5c/d64/db3 0 2024-03-11T19:55:58.536 INFO:tasks.workunit.client.0.smithi107.stdout:3/507: sync 2024-03-11T19:55:58.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/482: symlink d4/d1d/d1f/d28/d5d/lb4 0 2024-03-11T19:55:58.537 INFO:tasks.workunit.client.0.smithi107.stdout:3/508: symlink d1/d10/d21/d43/d48/d98/l9a 0 2024-03-11T19:55:58.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/483: symlink d4/d9/lb5 0 2024-03-11T19:55:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:9/484: mknod d4/d9/dae/d6d/db0/cb6 0 2024-03-11T19:55:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:9/485: mknod d4/d1d/d5c/d64/db3/cb7 0 2024-03-11T19:55:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:9/486: unlink d4/d1d/d4b/f88 0 2024-03-11T19:55:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/518: write d4/d7/f22 [637184,69870] 0 2024-03-11T19:55:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/487: link d4/d9/dae/d6d/l33 d4/d9/dae/d6d/db0/lb8 0 2024-03-11T19:55:58.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/488: creat d4/d1d/d5c/d97/fb9 x:0 0 0 2024-03-11T19:55:58.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/489: stat d4/d1c/d47/d8a/l8d 0 2024-03-11T19:55:58.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/490: rename d4/d1c/d47/d8a/l8d to d4/d1c/d47/lba 0 2024-03-11T19:55:58.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/491: creat d4/d1c/d37/fbb x:0 0 0 2024-03-11T19:55:58.561 INFO:tasks.workunit.client.0.smithi107.stdout:1/549: dread d6/d7e/d81/d27/d2c/f45 [0,4194304] 0 2024-03-11T19:55:58.562 INFO:tasks.workunit.client.0.smithi107.stdout:7/551: dread d6/d9/f4f [0,4194304] 0 2024-03-11T19:55:58.562 INFO:tasks.workunit.client.0.smithi107.stdout:7/552: truncate d6/da/d16/d5c/d7d/f51 4623337 0 2024-03-11T19:55:58.563 INFO:tasks.workunit.client.0.smithi107.stdout:1/550: symlink d6/d7e/da5/dac/d82/lb0 0 2024-03-11T19:55:58.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/551: creat d6/d7e/d81/d27/d2c/fb1 x:0 0 0 2024-03-11T19:55:58.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/552: fsync d6/d28/d60/f96 0 2024-03-11T19:55:58.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/553: chown d6/c49 97240 1 2024-03-11T19:55:58.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/554: rename d6/d7e/d81/d27/d2c/f45 to d6/d28/fb2 0 2024-03-11T19:55:58.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/519: dread d4/fc [0,4194304] 0 2024-03-11T19:55:58.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/520: write d4/d7/d1f/d2b/d24/d26/d29/f30 [4817169,16466] 0 2024-03-11T19:55:58.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/553: dread d6/da/d16/f98 [0,4194304] 0 2024-03-11T19:55:58.585 INFO:tasks.workunit.client.0.smithi107.stdout:3/509: dwrite d1/d10/d21/d43/d48/f8e [0,4194304] 0 2024-03-11T19:55:58.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/450: dwrite dd/f4c [0,4194304] 0 2024-03-11T19:55:58.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/451: fsync dd/f4c 0 2024-03-11T19:55:58.592 INFO:tasks.workunit.client.0.smithi107.stdout:4/521: rmdir d4/d7/d2f/d33 39 2024-03-11T19:55:58.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/435: dwrite d10/d19/f65 [0,4194304] 0 2024-03-11T19:55:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/436: creat d10/d16/d24/d85/f92 x:0 0 0 2024-03-11T19:55:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/437: readlink d10/d16/d1f/d51/l8f 0 2024-03-11T19:55:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/438: chown d10/d16/d2b/f59 185 1 2024-03-11T19:55:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/439: chown d10/d16/d24/f58 26127834 1 2024-03-11T19:55:58.596 INFO:tasks.workunit.client.0.smithi107.stdout:4/522: creat d4/d7/da0/fa1 x:0 0 0 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:5/576: sync 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/523: getdents d4/d7/db 0 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/524: write d4/d7/d2f/d3c/f73 [669430,55298] 0 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/518: sync 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/519: write de/f18 [3714924,6561] 0 2024-03-11T19:55:58.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/520: chown de/d11/c22 258714524 1 2024-03-11T19:55:58.603 INFO:tasks.workunit.client.0.smithi107.stdout:0/521: creat de/d11/d23/d32/d34/d39/f9f x:0 0 0 2024-03-11T19:55:58.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/492: dwrite d4/d1d/d1f/d78/f1b [0,4194304] 0 2024-03-11T19:55:58.612 INFO:tasks.workunit.client.0.smithi107.stdout:4/525: truncate d4/d7/d1f/d2b/d24/d26/d29/f3f 3606765 0 2024-03-11T19:55:58.612 INFO:tasks.workunit.client.0.smithi107.stdout:4/526: chown d4/d7/d1f/d2b/d24/d26/d29/d36/f7a 62496888 1 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/452: dread dd/d16/f19 [4194304,4194304] 0 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/453: dread - dd/de/f48 zero size 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/454: readlink dd/de/d1b/d2c/l38 0 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/455: creat dd/de/d1b/f83 x:0 0 0 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/456: chown dd/de/d24/d28/d47 248629220 1 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/493: creat d4/d9/d9f/fbc x:0 0 0 2024-03-11T19:55:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/457: truncate dd/de/d49/f5b 1380630 0 2024-03-11T19:55:58.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/527: truncate d4/d7/d1f/d2b/f56 4275586 0 2024-03-11T19:55:58.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/528: dread - d4/d7/d2f/d3c/f77 zero size 2024-03-11T19:55:58.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/529: stat d4/d7/d2f/d3c/l69 0 2024-03-11T19:55:58.618 INFO:tasks.workunit.client.0.smithi107.stdout:9/494: link d4/d9/dae/f77 d4/d1d/d5c/d97/fbd 0 2024-03-11T19:55:58.618 INFO:tasks.workunit.client.0.smithi107.stdout:9/495: dread - d4/d1c/d37/f75 zero size 2024-03-11T19:55:58.618 INFO:tasks.workunit.client.0.smithi107.stdout:0/522: dread de/d11/d23/d24/f69 [0,4194304] 0 2024-03-11T19:55:58.632 INFO:tasks.workunit.client.0.smithi107.stdout:0/523: dread de/d11/d23/d32/d34/d39/f53 [0,4194304] 0 2024-03-11T19:55:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/524: symlink de/d11/d23/d24/d4f/d89/la0 0 2024-03-11T19:55:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:0/525: symlink de/d11/d23/d24/la1 0 2024-03-11T19:55:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/526: mkdir de/d11/d74/da2 0 2024-03-11T19:55:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/527: fdatasync de/d11/d23/d26/f43 0 2024-03-11T19:55:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/528: write de/f6a [634121,2210] 0 2024-03-11T19:55:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/482: sync 2024-03-11T19:55:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:2/483: truncate d1/d1a/f74 205396 0 2024-03-11T19:55:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:2/484: fsync d1/f49 0 2024-03-11T19:55:58.647 INFO:tasks.workunit.client.0.smithi107.stdout:1/555: dwrite d6/d28/d60/f9b [0,4194304] 0 2024-03-11T19:55:58.647 INFO:tasks.workunit.client.0.smithi107.stdout:1/556: rename d6/d7e to d6/d7e/da5/dac/d62/da3/db3 22 2024-03-11T19:55:58.647 INFO:tasks.workunit.client.0.smithi107.stdout:1/557: chown d6/d7e/d81/d27/d2c/d32/c50 1 1 2024-03-11T19:55:58.647 INFO:tasks.workunit.client.0.smithi107.stdout:3/510: dread d1/d10/f57 [0,4194304] 0 2024-03-11T19:55:58.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/511: link d1/d10/l1c d1/d10/d13/d6c/d5d/d76/l9b 0 2024-03-11T19:55:58.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/554: dwrite d6/d9/d4e/fa4 [4194304,4194304] 0 2024-03-11T19:55:58.663 INFO:tasks.workunit.client.0.smithi107.stdout:3/512: rmdir d1/d10/d13/d6c/d5d/d99 0 2024-03-11T19:55:58.663 INFO:tasks.workunit.client.0.smithi107.stdout:3/513: chown d1/d10/d26/d17 402269 1 2024-03-11T19:55:58.663 INFO:tasks.workunit.client.0.smithi107.stdout:3/514: truncate d1/d10/d13/d4a/f93 636103 0 2024-03-11T19:55:58.665 INFO:tasks.workunit.client.0.smithi107.stdout:5/577: dwrite da/d14/d43/f6c [0,4194304] 0 2024-03-11T19:55:58.665 INFO:tasks.workunit.client.0.smithi107.stdout:8/458: dwrite dd/de/d49/f2b [0,4194304] 0 2024-03-11T19:55:58.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/459: symlink dd/de/d1b/d2c/l84 0 2024-03-11T19:55:58.671 INFO:tasks.workunit.client.0.smithi107.stdout:8/460: creat dd/d13/d32/d51/f85 x:0 0 0 2024-03-11T19:55:58.671 INFO:tasks.workunit.client.0.smithi107.stdout:8/461: fsync dd/de/d1b/d20/f6a 0 2024-03-11T19:55:58.674 INFO:tasks.workunit.client.0.smithi107.stdout:8/462: mkdir dd/de/d24/d86 0 2024-03-11T19:55:58.675 INFO:tasks.workunit.client.0.smithi107.stdout:4/530: dwrite d4/d7/d1f/d2b/d24/d26/d29/d36/f78 [0,4194304] 0 2024-03-11T19:55:58.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/463: read dd/d13/d18/f5c [95094,1617] 0 2024-03-11T19:55:58.699 INFO:tasks.workunit.client.0.smithi107.stdout:0/529: dwrite de/f16 [4194304,4194304] 0 2024-03-11T19:55:58.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/530: truncate de/d11/d23/d32/d34/d39/d3a/f5e 4668702 0 2024-03-11T19:55:58.704 INFO:tasks.workunit.client.0.smithi107.stdout:6/440: dwrite d10/d16/d1f/f25 [0,4194304] 0 2024-03-11T19:55:58.704 INFO:tasks.workunit.client.0.smithi107.stdout:6/441: creat d10/d16/d1f/d51/d76/f93 x:0 0 0 2024-03-11T19:55:58.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/442: creat d10/d16/d1f/d51/d7a/f94 x:0 0 0 2024-03-11T19:55:58.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/443: dread - d10/d16/d1f/d51/f87 zero size 2024-03-11T19:55:58.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/444: truncate d10/d16/f33 5225590 0 2024-03-11T19:55:58.723 INFO:tasks.workunit.client.0.smithi107.stdout:7/555: dwrite d6/f5f [0,4194304] 0 2024-03-11T19:55:58.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/515: dwrite d1/d10/f57 [4194304,4194304] 0 2024-03-11T19:55:58.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/516: chown d1/d10/d13/d6c/d2d/d4d/d91 6291 1 2024-03-11T19:55:58.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/578: dread f4 [0,4194304] 0 2024-03-11T19:55:58.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/485: write d1/d1d/f24 [398341,105665] 0 2024-03-11T19:55:58.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/486: readlink d1/d1a/d34/d4f/l93 0 2024-03-11T19:55:58.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/579: dread - da/d14/d18/d1e/d9b/d72/fac zero size 2024-03-11T19:55:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/517: getdents d1/d10/d4e 0 2024-03-11T19:55:58.762 INFO:tasks.workunit.client.0.smithi107.stdout:0/531: dwrite de/d7a/f85 [0,4194304] 0 2024-03-11T19:55:58.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/445: dread d10/d16/d24/d2e/f2f [0,4194304] 0 2024-03-11T19:55:58.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/531: dwrite d4/d7/d2f/f65 [0,4194304] 0 2024-03-11T19:55:58.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/496: dwrite d4/d1d/d5c/d64/f84 [0,4194304] 0 2024-03-11T19:55:58.763 INFO:tasks.workunit.client.0.smithi107.stdout:9/497: truncate d4/d9/f86 1336362 0 2024-03-11T19:55:58.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/532: mkdir de/d11/d74/da3 0 2024-03-11T19:55:58.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/533: write de/d11/d23/d24/f69 [1432773,178] 0 2024-03-11T19:55:58.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/534: truncate de/d11/d23/d32/d34/d39/d55/f95 942828 0 2024-03-11T19:55:58.768 INFO:tasks.workunit.client.0.smithi107.stdout:6/446: creat d10/d16/d1f/d51/d76/d7f/f95 x:0 0 0 2024-03-11T19:55:58.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/464: dwrite dd/de/d1b/d20/d5e/f80 [0,4194304] 0 2024-03-11T19:55:58.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/465: chown dd/de/d1b 0 1 2024-03-11T19:55:58.770 INFO:tasks.workunit.client.0.smithi107.stdout:0/535: unlink de/f59 0 2024-03-11T19:55:58.773 INFO:tasks.workunit.client.0.smithi107.stdout:6/447: creat d10/d3d/d54/f96 x:0 0 0 2024-03-11T19:55:58.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/556: dwrite d6/d21/f4c [0,4194304] 0 2024-03-11T19:55:58.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/536: unlink de/d11/d23/d32/f8f 0 2024-03-11T19:55:58.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/557: rename d6/da/l27 to d6/lad 0 2024-03-11T19:55:58.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/580: dwrite da/d14/d38/f78 [0,4194304] 0 2024-03-11T19:55:58.780 INFO:tasks.workunit.client.0.smithi107.stdout:7/558: rmdir d6/d9/d4e 39 2024-03-11T19:55:58.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/466: dread dd/de/d1b/d20/f43 [0,4194304] 0 2024-03-11T19:55:58.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/467: truncate dd/d13/f52 838696 0 2024-03-11T19:55:58.786 INFO:tasks.workunit.client.0.smithi107.stdout:5/581: creat da/d14/d18/fb1 x:0 0 0 2024-03-11T19:55:58.787 INFO:tasks.workunit.client.0.smithi107.stdout:7/559: mkdir d6/d9/dae 0 2024-03-11T19:55:58.787 INFO:tasks.workunit.client.0.smithi107.stdout:7/560: chown d6/da/d16/d5c/d7d/d1b/d76/c88 0 1 2024-03-11T19:55:58.788 INFO:tasks.workunit.client.0.smithi107.stdout:5/582: mkdir da/d14/d18/db2 0 2024-03-11T19:55:58.792 INFO:tasks.workunit.client.0.smithi107.stdout:3/518: dwrite d1/f6e [0,4194304] 0 2024-03-11T19:55:58.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/561: truncate d6/da/d16/f36 1921292 0 2024-03-11T19:55:58.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/519: write d1/d10/d13/d4a/f93 [168057,31725] 0 2024-03-11T19:55:58.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/532: write d4/d7/d1f/d2b/d24/d26/f84 [1365313,52336] 0 2024-03-11T19:55:58.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/533: chown d4/d7/d2f/f65 15029 1 2024-03-11T19:55:58.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/534: creat d4/d7/d1f/d2b/d19/fa2 x:0 0 0 2024-03-11T19:55:58.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/448: dread f3 [0,4194304] 0 2024-03-11T19:55:58.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/487: dwrite d1/d38/d75/f7d [0,4194304] 0 2024-03-11T19:55:58.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/488: fsync d1/d1a/d1e/d55/f87 0 2024-03-11T19:55:58.815 INFO:tasks.workunit.client.0.smithi107.stdout:6/449: link d10/d16/d1f/f28 d10/d16/d1f/d51/d76/d7f/f97 0 2024-03-11T19:55:58.815 INFO:tasks.workunit.client.0.smithi107.stdout:2/489: mkdir d1/d1a/d1e/d26/da6 0 2024-03-11T19:55:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/583: dread da/d14/d18/f27 [4194304,4194304] 0 2024-03-11T19:55:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/468: dwrite dd/d13/f52 [0,4194304] 0 2024-03-11T19:55:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/469: write dd/de/d1b/d20/d5e/f65 [76182,51921] 0 2024-03-11T19:55:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/470: write dd/de/f48 [411039,29921] 0 2024-03-11T19:55:58.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/471: fdatasync dd/de/d1b/d20/f29 0 2024-03-11T19:55:58.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/472: mkdir dd/de/d1b/d20/d56/d87 0 2024-03-11T19:55:58.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/473: write dd/de/d24/d28/f4a [1236717,81542] 0 2024-03-11T19:55:58.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/474: truncate dd/d16/f7f 446535 0 2024-03-11T19:55:58.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/475: dread - dd/d13/d18/f75 zero size 2024-03-11T19:55:58.840 INFO:tasks.workunit.client.0.smithi107.stdout:8/476: rmdir dd/de/d24/d28 39 2024-03-11T19:55:58.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/537: dread de/d11/d23/d32/d34/d39/d3a/f50 [0,4194304] 0 2024-03-11T19:55:58.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/498: dwrite d4/d1d/f3b [0,4194304] 0 2024-03-11T19:55:58.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/499: truncate d4/d9/d9f/fbc 30841 0 2024-03-11T19:55:58.857 INFO:tasks.workunit.client.0.smithi107.stdout:2/490: dread d1/d21/d35/f54 [0,4194304] 0 2024-03-11T19:55:58.859 INFO:tasks.workunit.client.0.smithi107.stdout:2/491: rmdir d1/d1a/d1e/d26/da6 0 2024-03-11T19:55:58.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/500: write d4/d1d/d1f/d28/fa2 [659517,58244] 0 2024-03-11T19:55:58.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/492: creat d1/d21/fa7 x:0 0 0 2024-03-11T19:55:58.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/450: dread d10/d16/d24/d2e/f46 [0,4194304] 0 2024-03-11T19:55:58.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/520: dwrite d1/d10/d21/d43/d48/f69 [0,4194304] 0 2024-03-11T19:55:58.866 INFO:tasks.workunit.client.0.smithi107.stdout:3/521: mkdir d1/d10/d4e/d73/d8d/d9c 0 2024-03-11T19:55:58.866 INFO:tasks.workunit.client.0.smithi107.stdout:3/522: dread - d1/d10/d21/d43/d4f/f59 zero size 2024-03-11T19:55:58.868 INFO:tasks.workunit.client.0.smithi107.stdout:3/523: creat d1/f9d x:0 0 0 2024-03-11T19:55:58.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/524: getdents d1/d10 0 2024-03-11T19:55:58.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/538: dread de/f18 [0,4194304] 0 2024-03-11T19:55:58.882 INFO:tasks.workunit.client.0.smithi107.stdout:5/584: write da/d31/d42/f11 [7503813,102627] 0 2024-03-11T19:55:58.882 INFO:tasks.workunit.client.0.smithi107.stdout:4/535: dwrite d4/d7/f98 [0,4194304] 0 2024-03-11T19:55:58.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/493: dread d1/d1a/d34/d4f/f83 [0,4194304] 0 2024-03-11T19:55:58.895 INFO:tasks.workunit.client.0.smithi107.stdout:5/585: unlink da/d14/d38/f78 0 2024-03-11T19:55:58.896 INFO:tasks.workunit.client.0.smithi107.stdout:5/586: mkdir da/d14/d18/d1e/d9b/db3 0 2024-03-11T19:55:58.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/477: read dd/f21 [3941552,30261] 0 2024-03-11T19:55:58.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/478: fsync dd/de/d49/f5b 0 2024-03-11T19:55:58.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/479: chown dd/de/d1b/d20/f43 29977873 1 2024-03-11T19:55:58.900 INFO:tasks.workunit.client.0.smithi107.stdout:8/480: truncate dd/d13/d18/f50 4850149 0 2024-03-11T19:55:58.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/494: dread d1/d21/d35/f5c [0,4194304] 0 2024-03-11T19:55:58.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/495: stat d1/d1d/d64/fa1 0 2024-03-11T19:55:58.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/481: truncate dd/f21 242573 0 2024-03-11T19:55:58.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/496: mknod d1/d38/d75/d92/ca8 0 2024-03-11T19:55:58.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/539: dread de/d11/d23/d26/f42 [0,4194304] 0 2024-03-11T19:55:58.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/497: chown d1/d21/d35/d67 7 1 2024-03-11T19:55:58.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/498: creat d1/d38/d75/fa9 x:0 0 0 2024-03-11T19:55:58.909 INFO:tasks.workunit.client.0.smithi107.stdout:2/499: fdatasync d1/d21/d35/f54 0 2024-03-11T19:55:58.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/540: read de/d11/d23/d32/d34/f6c [102238,92402] 0 2024-03-11T19:55:58.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/541: stat de/d11/d23/d32/d34/d39/c93 0 2024-03-11T19:55:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:2/500: mkdir d1/d1a/d34/d4f/d63/daa 0 2024-03-11T19:55:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/536: dread d4/d7/f2c [0,4194304] 0 2024-03-11T19:55:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/542: mkdir de/d11/d23/d24/da4 0 2024-03-11T19:55:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/501: creat d1/d1d/d64/d97/fab x:0 0 0 2024-03-11T19:55:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/502: dread - d1/d21/d35/fa5 zero size 2024-03-11T19:55:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/503: chown d1/d1a/d1e/l25 241 1 2024-03-11T19:55:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/504: creat d1/d21/d35/fac x:0 0 0 2024-03-11T19:55:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/543: creat de/d11/d74/fa5 x:0 0 0 2024-03-11T19:55:58.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/501: dwrite d4/d1c/d37/fbb [0,4194304] 0 2024-03-11T19:55:58.933 INFO:tasks.workunit.client.0.smithi107.stdout:2/505: mknod d1/d1a/d34/d4f/d63/cad 0 2024-03-11T19:55:58.933 INFO:tasks.workunit.client.0.smithi107.stdout:2/506: truncate d1/d1d/f24 1777147 0 2024-03-11T19:55:58.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/544: mknod de/d11/d23/d26/ca6 0 2024-03-11T19:55:58.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/451: dwrite d10/d16/d2b/d39/d6a/f83 [0,4194304] 0 2024-03-11T19:55:58.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/507: unlink d1/d1a/d1e/l25 0 2024-03-11T19:55:58.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/545: creat de/d58/fa7 x:0 0 0 2024-03-11T19:55:58.937 INFO:tasks.workunit.client.0.smithi107.stdout:6/452: creat d10/d3d/f98 x:0 0 0 2024-03-11T19:55:58.937 INFO:tasks.workunit.client.0.smithi107.stdout:6/453: chown d10/d16/d2b/d39/l63 4800876 1 2024-03-11T19:55:58.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/454: fdatasync d10/d16/d1f/d51/f87 0 2024-03-11T19:55:58.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/455: stat d10/d16/d1f/d51/d76/d7f/f95 0 2024-03-11T19:55:58.938 INFO:tasks.workunit.client.0.smithi107.stdout:2/508: symlink d1/d1a/d34/lae 0 2024-03-11T19:55:58.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/456: creat d10/d19/d3b/f99 x:0 0 0 2024-03-11T19:55:58.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/509: symlink d1/d1a/d34/d4f/d63/laf 0 2024-03-11T19:55:58.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/510: readlink d1/d1a/d34/d4f/l93 0 2024-03-11T19:55:58.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/511: write d1/d38/d75/d76/f98 [610101,57784] 0 2024-03-11T19:55:58.941 INFO:tasks.workunit.client.0.smithi107.stdout:2/512: link d1/f36 d1/d38/d75/fb0 0 2024-03-11T19:55:58.941 INFO:tasks.workunit.client.0.smithi107.stdout:0/546: write de/d11/d23/d32/d34/d39/d55/f95 [923065,52033] 0 2024-03-11T19:55:58.941 INFO:tasks.workunit.client.0.smithi107.stdout:0/547: write de/d11/d23/d32/d34/d39/d3a/d72/f98 [443258,92745] 0 2024-03-11T19:55:58.942 INFO:tasks.workunit.client.0.smithi107.stdout:0/548: write de/d11/f4c [3497471,20310] 0 2024-03-11T19:55:58.945 INFO:tasks.workunit.client.0.smithi107.stdout:0/549: truncate de/d11/d23/d26/f43 2364615 0 2024-03-11T19:55:58.945 INFO:tasks.workunit.client.0.smithi107.stdout:2/513: symlink d1/d1a/d34/d4f/lb1 0 2024-03-11T19:55:58.945 INFO:tasks.workunit.client.0.smithi107.stdout:2/514: creat d1/d1a/d1e/d26/fb2 x:0 0 0 2024-03-11T19:55:58.951 INFO:tasks.workunit.client.0.smithi107.stdout:6/457: creat d10/d16/d2b/f9a x:0 0 0 2024-03-11T19:55:58.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/587: dwrite da/d14/d38/d59/f64 [0,4194304] 0 2024-03-11T19:55:58.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/588: write da/d14/d18/d1e/d9b/d73/fa3 [431676,115464] 0 2024-03-11T19:55:58.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/502: dread d4/d9/dae/d6d/f2d [0,4194304] 0 2024-03-11T19:55:58.956 INFO:tasks.workunit.client.0.smithi107.stdout:5/589: truncate da/d14/d18/f2c 1723758 0 2024-03-11T19:55:58.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/458: mknod d10/d16/d1f/d51/c9b 0 2024-03-11T19:55:58.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/537: dwrite d4/d7/d1f/d2b/d19/f23 [4194304,4194304] 0 2024-03-11T19:55:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:8/482: dwrite dd/d16/f33 [0,4194304] 0 2024-03-11T19:55:58.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/483: mkdir dd/de/d49/d46/d88 0 2024-03-11T19:55:58.980 INFO:tasks.workunit.client.0.smithi107.stdout:8/484: write dd/d13/d18/f75 [452035,28001] 0 2024-03-11T19:55:58.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/550: dwrite de/d11/d23/d32/d34/d39/d3a/f5e [0,4194304] 0 2024-03-11T19:55:58.988 INFO:tasks.workunit.client.0.smithi107.stdout:0/551: symlink de/d11/d23/d24/d8d/la8 0 2024-03-11T19:55:58.988 INFO:tasks.workunit.client.0.smithi107.stdout:0/552: chown de/d11/d23/d26/f43 0 1 2024-03-11T19:55:58.989 INFO:tasks.workunit.client.0.smithi107.stdout:0/553: mknod de/d11/d74/ca9 0 2024-03-11T19:55:59.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:55:58 smithi122 ceph-mon[36397]: pgmap v79: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 146 MiB/s wr, 661 op/s 2024-03-11T19:55:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:0/554: dread de/d11/f45 [0,4194304] 0 2024-03-11T19:55:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:0/555: write de/d11/d23/d32/d34/d39/f9f [588720,76153] 0 2024-03-11T19:55:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:0/556: truncate de/d11/d23/d26/f29 7991 0 2024-03-11T19:55:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:0/557: write de/d11/d23/d24/f60 [417411,129362] 0 2024-03-11T19:55:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:0/558: write de/d11/d23/f71 [5123647,115468] 0 2024-03-11T19:55:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:5/590: dread da/d14/d38/d59/f69 [0,4194304] 0 2024-03-11T19:55:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/538: dwrite d4/d7/d1f/d2b/d24/d26/d29/f3f [4194304,4194304] 0 2024-03-11T19:55:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:0/559: truncate de/d11/d23/d24/f69 1197132 0 2024-03-11T19:55:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:4/539: mknod d4/d7/d1f/d2b/d24/ca3 0 2024-03-11T19:55:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:4/540: truncate d4/d7/d1f/d2b/d19/f2a 2283615 0 2024-03-11T19:55:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:4/541: chown d4/d7/d1f/d2b/d19/f2a 68941 1 2024-03-11T19:55:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/459: dread d10/d16/f84 [0,4194304] 0 2024-03-11T19:55:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/460: stat d10/d19/f89 0 2024-03-11T19:55:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:2/515: dwrite d1/d38/d75/f7d [4194304,4194304] 0 2024-03-11T19:55:59.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/516: readlink d1/l2 0 2024-03-11T19:55:59.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/542: creat d4/d7/d1f/d2b/d24/d26/d29/fa4 x:0 0 0 2024-03-11T19:55:59.028 INFO:tasks.workunit.client.0.smithi107.stdout:2/517: rename d1/d38/d75/d92 to d1/d1a/d34/d4f/db3 0 2024-03-11T19:55:59.028 INFO:tasks.workunit.client.0.smithi107.stdout:2/518: fdatasync d1/d21/d35/f8d 0 2024-03-11T19:55:59.029 INFO:tasks.workunit.client.0.smithi107.stdout:2/519: read d1/d21/d35/f54 [2625389,79240] 0 2024-03-11T19:55:59.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/558: sync 2024-03-11T19:55:59.031 INFO:tasks.workunit.client.0.smithi107.stdout:7/562: sync 2024-03-11T19:55:59.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/520: symlink d1/d1a/d8e/lb4 0 2024-03-11T19:55:59.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/563: symlink d6/da/d16/d91/d97/d9b/laf 0 2024-03-11T19:55:59.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/521: chown d1/c3b 0 1 2024-03-11T19:55:59.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:55:58 smithi145 ceph-mon[36730]: pgmap v79: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 146 MiB/s wr, 661 op/s 2024-03-11T19:55:59.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/522: creat d1/d1a/d34/fb5 x:0 0 0 2024-03-11T19:55:59.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/523: chown d1/d1a/d34/d4f/d63/f6f 159835 1 2024-03-11T19:55:59.035 INFO:tasks.workunit.client.0.smithi107.stdout:1/559: rename d6/d19/c59 to d6/d7e/d81/cb4 0 2024-03-11T19:55:59.036 INFO:tasks.workunit.client.0.smithi107.stdout:7/564: creat d6/da/d16/d5c/d7d/d1b/d76/d80/fb0 x:0 0 0 2024-03-11T19:55:59.036 INFO:tasks.workunit.client.0.smithi107.stdout:1/560: creat d6/d19/fb5 x:0 0 0 2024-03-11T19:55:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/524: unlink d1/d21/d35/fac 0 2024-03-11T19:55:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/525: read - d1/d21/d35/fa5 zero size 2024-03-11T19:55:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/591: dread da/d14/f3a [0,4194304] 0 2024-03-11T19:55:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/561: mknod d6/d28/d60/d95/cb6 0 2024-03-11T19:55:59.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/565: link d6/da/d16/d5c/d7d/c12 d6/da/d16/d5c/d7d/d60/d61/d78/cb1 0 2024-03-11T19:55:59.038 INFO:tasks.workunit.client.0.smithi107.stdout:1/562: mknod d6/d7e/d81/cb7 0 2024-03-11T19:55:59.039 INFO:tasks.workunit.client.0.smithi107.stdout:7/566: creat d6/da/d16/d91/d97/fb2 x:0 0 0 2024-03-11T19:55:59.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:55:58 smithi107 ceph-mon[31712]: pgmap v79: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 146 MiB/s wr, 661 op/s 2024-03-11T19:55:59.047 INFO:tasks.workunit.client.0.smithi107.stdout:6/461: write d10/d19/d38/f5c [290180,18585] 0 2024-03-11T19:55:59.049 INFO:tasks.workunit.client.0.smithi107.stdout:6/462: mkdir d10/d16/d1f/d51/d9c 0 2024-03-11T19:55:59.053 INFO:tasks.workunit.client.0.smithi107.stdout:6/463: rename d10/d16/d24/d85 to d10/d16/d24/d2e/d9d 0 2024-03-11T19:55:59.058 INFO:tasks.workunit.client.0.smithi107.stdout:6/464: rmdir d10/d16/d24 39 2024-03-11T19:55:59.060 INFO:tasks.workunit.client.0.smithi107.stdout:0/560: dwrite f3 [0,4194304] 0 2024-03-11T19:55:59.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/563: link d6/d7e/d81/d27/d2c/d32/l3a d6/d7e/d88/lb8 0 2024-03-11T19:55:59.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/592: dread da/d14/d38/d59/f69 [0,4194304] 0 2024-03-11T19:55:59.062 INFO:tasks.workunit.client.0.smithi107.stdout:7/567: write d6/f5f [3982028,28110] 0 2024-03-11T19:55:59.062 INFO:tasks.workunit.client.0.smithi107.stdout:0/561: truncate de/f16 1175726 0 2024-03-11T19:55:59.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/593: truncate da/d6e/f82 3825901 0 2024-03-11T19:55:59.063 INFO:tasks.workunit.client.0.smithi107.stdout:6/465: symlink d10/d19/d3b/d70/l9e 0 2024-03-11T19:55:59.065 INFO:tasks.workunit.client.0.smithi107.stdout:6/466: fsync d10/d16/f2d 0 2024-03-11T19:55:59.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/562: symlink de/d11/d23/d32/d34/d39/d3a/laa 0 2024-03-11T19:55:59.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/563: write de/d11/d23/d32/d34/d39/d3a/f66 [706481,39665] 0 2024-03-11T19:55:59.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/564: read - de/d58/f61 zero size 2024-03-11T19:55:59.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/594: symlink da/d14/d43/d9e/lb4 0 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/595: stat da/d14/d38/f3f 0 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/596: truncate da/d14/d18/d1e/d9b/d73/faa 908855 0 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/597: fdatasync da/d31/d42/f11 0 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/564: mknod d6/cb9 0 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/565: chown d6/d7e/d81/l52 95 1 2024-03-11T19:55:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:2/526: dread d1/d1d/d64/f99 [0,4194304] 0 2024-03-11T19:55:59.067 INFO:tasks.workunit.client.0.smithi107.stdout:1/566: creat d6/d7e/da5/dac/d62/da3/fba x:0 0 0 2024-03-11T19:55:59.067 INFO:tasks.workunit.client.0.smithi107.stdout:1/567: stat d6/d7e/d81/d27/d2c/l2d 0 2024-03-11T19:55:59.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/568: rmdir d6/d7e/da5 39 2024-03-11T19:55:59.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/565: truncate fd 626852 0 2024-03-11T19:55:59.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/598: rename da/d14/d18/d6d/l8d to da/d14/d18/d6d/lb5 0 2024-03-11T19:55:59.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/599: fdatasync da/d14/d43/f6c 0 2024-03-11T19:55:59.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/569: mkdir d6/d28/d60/d95/dbb 0 2024-03-11T19:55:59.075 INFO:tasks.workunit.client.0.smithi107.stdout:0/566: creat de/d11/d23/d24/d83/fab x:0 0 0 2024-03-11T19:55:59.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/567: mknod de/d58/cac 0 2024-03-11T19:55:59.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/568: write de/d11/d23/d24/d4f/f78 [1082302,80117] 0 2024-03-11T19:55:59.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/569: getdents de/d11/d74/da2 0 2024-03-11T19:55:59.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/527: dread d1/f10 [0,4194304] 0 2024-03-11T19:55:59.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/528: getdents d1/d1a 0 2024-03-11T19:55:59.092 INFO:tasks.workunit.client.0.smithi107.stdout:2/529: write d1/d1a/d34/fb5 [933287,30409] 0 2024-03-11T19:55:59.092 INFO:tasks.workunit.client.0.smithi107.stdout:2/530: truncate d1/d1a/d1e/d41/f51 1252640 0 2024-03-11T19:55:59.093 INFO:tasks.workunit.client.0.smithi107.stdout:2/531: unlink d1/f49 0 2024-03-11T19:55:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/532: read d1/d1a/d34/fb5 [241559,44051] 0 2024-03-11T19:55:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:1/570: dread d6/d7e/da5/dac/d62/f65 [0,4194304] 0 2024-03-11T19:55:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:1/571: chown d6/d28/d60/f89 226182882 1 2024-03-11T19:55:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/533: mkdir d1/d1a/d34/d4f/d63/db6 0 2024-03-11T19:55:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:7/568: dwrite d6/f2f [8388608,4194304] 0 2024-03-11T19:55:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/572: unlink d6/c49 0 2024-03-11T19:55:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/569: truncate d6/da/d16/f2e 3223616 0 2024-03-11T19:55:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:7/570: stat d6/da/d4d 0 2024-03-11T19:55:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/534: truncate d1/d1d/f5b 851515 0 2024-03-11T19:55:59.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/571: creat d6/da/d16/d5c/d7d/d60/d61/fb3 x:0 0 0 2024-03-11T19:55:59.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/572: mknod d6/d9/cb4 0 2024-03-11T19:55:59.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/467: dwrite d10/f71 [0,4194304] 0 2024-03-11T19:55:59.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/468: creat d10/d16/d1f/d51/d76/f9f x:0 0 0 2024-03-11T19:55:59.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/469: symlink d10/d16/d2b/d39/d6a/d6e/la0 0 2024-03-11T19:55:59.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/573: dread d6/d5b/f78 [0,4194304] 0 2024-03-11T19:55:59.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/570: dread f5 [0,4194304] 0 2024-03-11T19:55:59.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/600: dwrite da/d14/d38/d46/fa1 [0,4194304] 0 2024-03-11T19:55:59.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/601: link da/d14/d18/f27 da/d14/d18/d1e/fb6 0 2024-03-11T19:55:59.130 INFO:tasks.workunit.client.0.smithi107.stdout:5/602: symlink da/d14/d18/d1e/d9b/lb7 0 2024-03-11T19:55:59.131 INFO:tasks.workunit.client.0.smithi107.stdout:3/525: sync 2024-03-11T19:55:59.134 INFO:tasks.workunit.client.0.smithi107.stdout:3/526: unlink d1/d45/f63 0 2024-03-11T19:55:59.163 INFO:tasks.workunit.client.0.smithi107.stdout:2/535: dread d1/f40 [0,4194304] 0 2024-03-11T19:55:59.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/536: creat d1/d1a/d34/d4f/db3/fb7 x:0 0 0 2024-03-11T19:55:59.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/537: getdents d1/d38 0 2024-03-11T19:55:59.170 INFO:tasks.workunit.client.0.smithi107.stdout:7/573: dwrite d6/da/d16/f2b [0,4194304] 0 2024-03-11T19:55:59.171 INFO:tasks.workunit.client.0.smithi107.stdout:7/574: mknod d6/d9/dae/cb5 0 2024-03-11T19:55:59.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/575: symlink d6/da/d16/d5c/d7d/d60/lb6 0 2024-03-11T19:55:59.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/576: write d6/da/d16/d5c/d7d/d60/d61/d78/f8d [484044,118308] 0 2024-03-11T19:55:59.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/603: dwrite da/d14/d38/d46/f9f [0,4194304] 0 2024-03-11T19:55:59.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/577: rename d6/d9/l34 to d6/da/d49/lb7 0 2024-03-11T19:55:59.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/604: creat da/d14/d43/d7f/d98/fb8 x:0 0 0 2024-03-11T19:55:59.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/605: write da/d14/d43/d47/f91 [1102269,21632] 0 2024-03-11T19:55:59.181 INFO:tasks.workunit.client.0.smithi107.stdout:5/606: unlink da/d14/l5c 0 2024-03-11T19:55:59.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/527: dwrite d1/d10/d21/d43/d48/f8e [0,4194304] 0 2024-03-11T19:55:59.182 INFO:tasks.workunit.client.0.smithi107.stdout:5/607: creat da/d14/d18/d1e/d9b/fb9 x:0 0 0 2024-03-11T19:55:59.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/608: dread - da/d14/d18/fb1 zero size 2024-03-11T19:55:59.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/609: fdatasync da/f32 0 2024-03-11T19:55:59.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/610: read da/d14/d38/d59/f69 [2667662,14183] 0 2024-03-11T19:55:59.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/528: creat d1/d10/d21/d43/d48/d98/f9e x:0 0 0 2024-03-11T19:55:59.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/611: link da/d14/d18/c9d da/d14/d18/d1e/cba 0 2024-03-11T19:55:59.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/612: dread - da/d14/d38/fa6 zero size 2024-03-11T19:55:59.186 INFO:tasks.workunit.client.0.smithi107.stdout:5/613: rename da/d14/d38/d46/f61 to da/d14/d43/d9e/dad/fbb 0 2024-03-11T19:55:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/614: symlink da/d14/d43/lbc 0 2024-03-11T19:55:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/615: truncate da/d14/f79 140816 0 2024-03-11T19:55:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/616: chown da/d14/d18/d1e/c34 162447 1 2024-03-11T19:55:59.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/538: read d1/d1a/d1e/d26/f48 [3077253,60507] 0 2024-03-11T19:55:59.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/539: truncate d1/d21/fa7 886861 0 2024-03-11T19:55:59.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/540: chown d1/d1a/d1e/d26/l52 1268722738 1 2024-03-11T19:55:59.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/541: truncate d1/d1a/d34/fb5 1535298 0 2024-03-11T19:55:59.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/542: mknod d1/d1a/d34/d4f/d63/daa/cb8 0 2024-03-11T19:55:59.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/543: truncate d1/d1a/f2f 7791545 0 2024-03-11T19:55:59.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/544: rename d1/d1a/d1e/l9d to d1/d1a/d34/lb9 0 2024-03-11T19:55:59.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/545: write d1/d38/d75/fa9 [277712,126318] 0 2024-03-11T19:55:59.203 INFO:tasks.workunit.client.0.smithi107.stdout:7/578: read d6/d9/d4e/f59 [3768664,57434] 0 2024-03-11T19:55:59.204 INFO:tasks.workunit.client.0.smithi107.stdout:0/571: dwrite de/d11/d23/f71 [4194304,4194304] 0 2024-03-11T19:55:59.204 INFO:tasks.workunit.client.0.smithi107.stdout:0/572: truncate de/f40 548892 0 2024-03-11T19:55:59.204 INFO:tasks.workunit.client.0.smithi107.stdout:0/573: readlink de/d11/d23/d24/d4f/d89/la0 0 2024-03-11T19:55:59.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/574: link de/d11/d23/d24/c90 de/d11/d23/d32/d34/d39/cad 0 2024-03-11T19:55:59.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/529: dwrite d1/d10/d13/d4a/d4c/f2f [8388608,4194304] 0 2024-03-11T19:55:59.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/574: dwrite d6/d7e/d88/f91 [0,4194304] 0 2024-03-11T19:55:59.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/575: fdatasync d6/d5b/f78 0 2024-03-11T19:55:59.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/576: creat d6/d86/fbc x:0 0 0 2024-03-11T19:55:59.232 INFO:tasks.workunit.client.0.smithi107.stdout:6/470: dwrite d10/d19/d3b/f99 [0,4194304] 0 2024-03-11T19:55:59.232 INFO:tasks.workunit.client.0.smithi107.stdout:6/471: fsync d10/d16/d1f/f28 0 2024-03-11T19:55:59.242 INFO:tasks.workunit.client.0.smithi107.stdout:0/575: rename de/d11/d23/d32/d34/d39/d3a/d72/d99 to de/d11/d23/d24/da4/dae 0 2024-03-11T19:55:59.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/503: sync 2024-03-11T19:55:59.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/485: sync 2024-03-11T19:55:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:4/543: sync 2024-03-11T19:55:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:9/504: write d4/d1d/d1f/d28/d2c/f34 [4285814,48118] 0 2024-03-11T19:55:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/577: rmdir d6/d7e/d81/d27/d2c 39 2024-03-11T19:55:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/578: truncate d6/d28/f7c 738078 0 2024-03-11T19:55:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/579: readlink d6/d7e/d88/lb8 0 2024-03-11T19:55:59.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/544: mkdir d4/d7/d1f/d2b/d19/d75/d8e/da5 0 2024-03-11T19:55:59.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/580: rename d6/d28/d60/d95/fae to d6/d7e/da5/dac/fbd 0 2024-03-11T19:55:59.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/581: fsync d6/d7e/d81/d27/d2c/f79 0 2024-03-11T19:55:59.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/617: dread da/f71 [0,4194304] 0 2024-03-11T19:55:59.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/618: chown da/d14/d38/d59/f69 107629551 1 2024-03-11T19:55:59.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/546: dwrite d1/d1d/d64/f8a [0,4194304] 0 2024-03-11T19:55:59.251 INFO:tasks.workunit.client.0.smithi107.stdout:9/505: unlink d4/f27 0 2024-03-11T19:55:59.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/582: symlink d6/d7e/d81/d27/lbe 0 2024-03-11T19:55:59.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/545: creat d4/d7/d1f/d2b/d19/d75/d8e/da5/fa6 x:0 0 0 2024-03-11T19:55:59.255 INFO:tasks.workunit.client.0.smithi107.stdout:5/619: mkdir da/d14/d38/d59/dbd 0 2024-03-11T19:55:59.255 INFO:tasks.workunit.client.0.smithi107.stdout:2/547: unlink d1/d1a/d1e/l37 0 2024-03-11T19:55:59.255 INFO:tasks.workunit.client.0.smithi107.stdout:2/548: write d1/d1a/d34/d4f/f5d [4325789,30012] 0 2024-03-11T19:55:59.256 INFO:tasks.workunit.client.0.smithi107.stdout:1/583: symlink d6/d7e/da5/lbf 0 2024-03-11T19:55:59.256 INFO:tasks.workunit.client.0.smithi107.stdout:1/584: stat d6/f68 0 2024-03-11T19:55:59.256 INFO:tasks.workunit.client.0.smithi107.stdout:1/585: write d6/d86/fbc [552068,48634] 0 2024-03-11T19:55:59.264 INFO:tasks.workunit.client.0.smithi107.stdout:7/579: dwrite d6/d9/f85 [0,4194304] 0 2024-03-11T19:55:59.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/620: symlink da/d14/d43/d9e/dad/lbe 0 2024-03-11T19:55:59.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/580: rename d6/da/d49/fa0 to d6/d9/dae/fb8 0 2024-03-11T19:55:59.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/576: dread de/d11/d23/d32/d34/d39/f5a [0,4194304] 0 2024-03-11T19:55:59.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/577: read - de/d11/d23/d32/d34/f36 zero size 2024-03-11T19:55:59.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/578: chown de/d7a 1803609 1 2024-03-11T19:55:59.268 INFO:tasks.workunit.client.0.smithi107.stdout:1/586: mkdir d6/d86/dc0 0 2024-03-11T19:55:59.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/587: rename d6/d28/d60 to d6/d7e/d81/d27/d2c/d32/dc1 0 2024-03-11T19:55:59.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/588: chown d6/d7e/d81/d27/d2c/d32/dc1/d95 389822172 1 2024-03-11T19:55:59.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/589: chown d6 1565 1 2024-03-11T19:55:59.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/590: write d6/d7e/d81/f24 [432353,129680] 0 2024-03-11T19:55:59.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/591: chown d6/d7e/d81/l69 43 1 2024-03-11T19:55:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:5/621: link da/d31/d42/c60 da/d14/d18/d1e/d9b/d73/cbf 0 2024-03-11T19:55:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:7/581: write d6/d21/f3a [873646,118063] 0 2024-03-11T19:55:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:3/530: dwrite d1/d10/d21/d43/d48/f50 [0,4194304] 0 2024-03-11T19:55:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:3/531: dread - d1/d10/d21/d43/d4f/f59 zero size 2024-03-11T19:55:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:5/622: creat da/d14/d18/d1e/d9b/fc0 x:0 0 0 2024-03-11T19:55:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/532: creat d1/d10/d21/d43/d48/f9f x:0 0 0 2024-03-11T19:55:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/623: mkdir da/d14/d96/dc1 0 2024-03-11T19:55:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/624: chown da/l99 5581 1 2024-03-11T19:55:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/625: write da/f86 [799527,63085] 0 2024-03-11T19:55:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/626: readlink da/d31/d42/l20 0 2024-03-11T19:55:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/627: chown da/d14/d43/f6c 2587849 1 2024-03-11T19:55:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/628: stat da/d14/d18/d1e/c34 0 2024-03-11T19:55:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/472: write d10/f14 [3354920,95965] 0 2024-03-11T19:55:59.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/533: rmdir d1/d10/d13/d6c/d2d/d4d 39 2024-03-11T19:55:59.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/592: write d6/d7e/da5/dac/d62/f65 [2122419,69061] 0 2024-03-11T19:55:59.318 INFO:tasks.workunit.client.0.smithi107.stdout:5/629: dread da/d31/d42/f28 [4194304,4194304] 0 2024-03-11T19:55:59.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/506: dwrite d4/d1d/d5c/d97/fb9 [0,4194304] 0 2024-03-11T19:55:59.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/579: dwrite de/d11/d23/d32/d34/d39/d3a/d72/f7d [0,4194304] 0 2024-03-11T19:55:59.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/580: truncate de/d11/d23/d32/d34/f9b 22428 0 2024-03-11T19:55:59.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/581: chown de/d11/d23/f76 6915248 1 2024-03-11T19:55:59.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/582: fdatasync de/d11/f91 0 2024-03-11T19:55:59.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/630: getdents da/d14/d18/d1e/d9b 0 2024-03-11T19:55:59.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/631: fdatasync da/d14/d43/f6c 0 2024-03-11T19:55:59.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/549: dwrite d1/d1a/f7c [0,4194304] 0 2024-03-11T19:55:59.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/486: dwrite dd/de/d1b/f7a [0,4194304] 0 2024-03-11T19:55:59.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/487: chown dd/de/d49/d46 0 1 2024-03-11T19:55:59.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/488: chown dd/de/d1b/d20/f43 1572 1 2024-03-11T19:55:59.344 INFO:tasks.workunit.client.0.smithi107.stdout:7/582: dwrite d6/da/d16/f2e [0,4194304] 0 2024-03-11T19:55:59.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/632: creat da/d14/d43/d7f/fc2 x:0 0 0 2024-03-11T19:55:59.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/633: dread - da/d14/d38/d46/faf zero size 2024-03-11T19:55:59.346 INFO:tasks.workunit.client.0.smithi107.stdout:6/473: rename d10/d16/d2b/l45 to d10/d16/d1f/d51/d7a/la1 0 2024-03-11T19:55:59.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/534: dwrite d1/d10/d4e/d73/d8d/f95 [0,4194304] 0 2024-03-11T19:55:59.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/507: write d4/d1d/d5c/d97/fb9 [1471831,28410] 0 2024-03-11T19:55:59.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/508: fdatasync d4/d1d/d1f/d28/f42 0 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/583: stat d6/da/c33 0 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/489: creat dd/d13/d32/f89 x:0 0 0 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/490: chown dd 231944404 1 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/535: mkdir d1/d10/d2a/d7e/da0 0 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/536: dread - d1/d10/d13/d4a/d4c/d49/f54 zero size 2024-03-11T19:55:59.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/537: write d1/d10/d13/d6c/d2d/d4d/f53 [686702,6096] 0 2024-03-11T19:55:59.361 INFO:tasks.workunit.client.0.smithi107.stdout:5/634: truncate da/f32 202456 0 2024-03-11T19:55:59.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/509: mknod d4/d1d/d1f/d28/cbe 0 2024-03-11T19:55:59.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/510: creat d4/fbf x:0 0 0 2024-03-11T19:55:59.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/511: write d4/d9/dae/f94 [17633,102132] 0 2024-03-11T19:55:59.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/584: rmdir d6/da/d49 39 2024-03-11T19:55:59.367 INFO:tasks.workunit.client.0.smithi107.stdout:5/635: creat da/d14/d38/d59/fc3 x:0 0 0 2024-03-11T19:55:59.369 INFO:tasks.workunit.client.0.smithi107.stdout:9/512: dread - d4/d9/dae/d6d/f68 zero size 2024-03-11T19:55:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/585: mkdir d6/da/d16/d5c/d7d/d60/d61/db9 0 2024-03-11T19:55:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/586: fsync d6/da/d16/f6f 0 2024-03-11T19:55:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/587: write d6/da/d4d/f71 [556929,110427] 0 2024-03-11T19:55:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/588: read - d6/d9/d57/f9e zero size 2024-03-11T19:55:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/538: mkdir d1/d10/d13/da1 0 2024-03-11T19:55:59.378 INFO:tasks.workunit.client.0.smithi107.stdout:5/636: link da/d14/d18/l41 da/lc4 0 2024-03-11T19:55:59.378 INFO:tasks.workunit.client.0.smithi107.stdout:9/513: creat d4/d1d/d1f/d78/fc0 x:0 0 0 2024-03-11T19:55:59.378 INFO:tasks.workunit.client.0.smithi107.stdout:9/514: chown d4/d1d/d1f/c35 220966 1 2024-03-11T19:55:59.379 INFO:tasks.workunit.client.0.smithi107.stdout:7/589: getdents d6/da/d16/d91/d97 0 2024-03-11T19:55:59.380 INFO:tasks.workunit.client.0.smithi107.stdout:5/637: getdents da/d14/d43/d7f/d98 0 2024-03-11T19:55:59.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/638: mknod da/d14/d96/dc1/cc5 0 2024-03-11T19:55:59.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/639: getdents da/d14/d18/db2 0 2024-03-11T19:55:59.387 INFO:tasks.workunit.client.0.smithi107.stdout:5/640: mknod da/d14/d18/d1e/d9b/cc6 0 2024-03-11T19:55:59.387 INFO:tasks.workunit.client.0.smithi107.stdout:5/641: creat da/d14/d43/d47/fc7 x:0 0 0 2024-03-11T19:55:59.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/474: rename d10/d16/d1f/d51/d76/d7f/f95 to d10/d3d/d54/fa2 0 2024-03-11T19:55:59.412 INFO:tasks.workunit.client.0.smithi107.stdout:9/515: dread d4/d9/dae/d6d/f2d [0,4194304] 0 2024-03-11T19:55:59.412 INFO:tasks.workunit.client.0.smithi107.stdout:7/590: dread d6/f5f [0,4194304] 0 2024-03-11T19:55:59.412 INFO:tasks.workunit.client.0.smithi107.stdout:9/516: creat d4/d1d/d1f/d28/fc1 x:0 0 0 2024-03-11T19:55:59.412 INFO:tasks.workunit.client.0.smithi107.stdout:9/517: stat d4/d1c/d37/l95 0 2024-03-11T19:55:59.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/491: write dd/d13/d32/d51/f61 [164609,112258] 0 2024-03-11T19:55:59.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/550: dwrite d1/d1d/f5b [0,4194304] 0 2024-03-11T19:55:59.413 INFO:tasks.workunit.client.0.smithi107.stdout:7/591: creat d6/da/d16/d5c/d7d/d60/fba x:0 0 0 2024-03-11T19:55:59.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/518: creat d4/d9/dae/d6d/db0/fc2 x:0 0 0 2024-03-11T19:55:59.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/519: write d4/d1d/d1f/d28/f6e [5143009,86075] 0 2024-03-11T19:55:59.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/520: chown d4/f54 6 1 2024-03-11T19:55:59.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/551: rmdir d1/d38 39 2024-03-11T19:55:59.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/552: fdatasync d1/d1a/d34/d4f/f83 0 2024-03-11T19:55:59.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/539: dwrite d1/d10/d4e/d73/f20 [0,4194304] 0 2024-03-11T19:55:59.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/540: chown d1/d10/d21/d43/d48/f69 2265 1 2024-03-11T19:55:59.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/583: dwrite de/d11/f35 [0,4194304] 0 2024-03-11T19:55:59.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/584: creat de/d11/d23/d32/faf x:0 0 0 2024-03-11T19:55:59.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/541: link d1/d10/d21/d43/d48/d98/f9e d1/d10/d4e/fa2 0 2024-03-11T19:55:59.424 INFO:tasks.workunit.client.0.smithi107.stdout:0/585: creat de/d11/fb0 x:0 0 0 2024-03-11T19:55:59.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/586: rmdir de 39 2024-03-11T19:55:59.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/587: chown de/d11/d23/d32/d34/d39/d55/f95 41023 1 2024-03-11T19:55:59.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/592: dread d6/d21/f4c [0,4194304] 0 2024-03-11T19:55:59.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/593: dread - d6/d9/d57/f9e zero size 2024-03-11T19:55:59.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/642: dwrite da/d14/f5e [0,4194304] 0 2024-03-11T19:55:59.437 INFO:tasks.workunit.client.0.smithi107.stdout:5/643: truncate da/d14/d18/d1e/d9b/d73/fae 54304 0 2024-03-11T19:55:59.437 INFO:tasks.workunit.client.0.smithi107.stdout:5/644: read da/f32 [185186,122880] 0 2024-03-11T19:55:59.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/588: creat de/d11/d74/da2/fb1 x:0 0 0 2024-03-11T19:55:59.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/594: rename d6/da/d16/d5c/d7d/f51 to d6/da/fbb 0 2024-03-11T19:55:59.450 INFO:tasks.workunit.client.0.smithi107.stdout:9/521: dread d4/d9/d4d/f8e [0,4194304] 0 2024-03-11T19:55:59.450 INFO:tasks.workunit.client.0.smithi107.stdout:8/492: dwrite dd/d16/f7f [0,4194304] 0 2024-03-11T19:55:59.451 INFO:tasks.workunit.client.0.smithi107.stdout:5/645: dread da/d14/d38/d59/f1d [0,4194304] 0 2024-03-11T19:55:59.451 INFO:tasks.workunit.client.0.smithi107.stdout:5/646: dread - da/d14/d18/d44/da5/fa8 zero size 2024-03-11T19:55:59.452 INFO:tasks.workunit.client.0.smithi107.stdout:5/647: dread - da/d14/d38/d59/d5f/f80 zero size 2024-03-11T19:55:59.452 INFO:tasks.workunit.client.0.smithi107.stdout:5/648: read - da/d14/d18/fb1 zero size 2024-03-11T19:55:59.454 INFO:tasks.workunit.client.0.smithi107.stdout:5/649: unlink da/d14/d18/d1e/d9b/d73/f92 0 2024-03-11T19:55:59.454 INFO:tasks.workunit.client.0.smithi107.stdout:9/522: getdents d4/d9/d4d 0 2024-03-11T19:55:59.454 INFO:tasks.workunit.client.0.smithi107.stdout:9/523: dread - d4/d9/dae/d6d/f68 zero size 2024-03-11T19:55:59.455 INFO:tasks.workunit.client.0.smithi107.stdout:9/524: write d4/d9/f86 [1337220,12124] 0 2024-03-11T19:55:59.458 INFO:tasks.workunit.client.0.smithi107.stdout:5/650: rmdir da/d14/d43/d7f 39 2024-03-11T19:55:59.460 INFO:tasks.workunit.client.0.smithi107.stdout:5/651: mkdir da/d14/d18/d1e/d9b/d72/dc8 0 2024-03-11T19:55:59.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/652: fdatasync da/d14/d43/d9e/dad/fbb 0 2024-03-11T19:55:59.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/653: dread - da/d14/d38/fa6 zero size 2024-03-11T19:55:59.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/493: write dd/f69 [3591372,2263] 0 2024-03-11T19:55:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:3/542: dwrite d1/d10/d26/f25 [0,4194304] 0 2024-03-11T19:55:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:3/543: symlink d1/d10/d4e/la3 0 2024-03-11T19:55:59.472 INFO:tasks.workunit.client.0.smithi107.stdout:7/595: write d6/f2f [6900729,88772] 0 2024-03-11T19:55:59.476 INFO:tasks.workunit.client.0.smithi107.stdout:7/596: write d6/da/d16/d5c/f58 [1927996,117334] 0 2024-03-11T19:55:59.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/597: creat d6/da/fbc x:0 0 0 2024-03-11T19:55:59.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/598: truncate d6/f17 1018773 0 2024-03-11T19:55:59.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/494: rename dd/de/d49/d46/l6f to dd/d13/d18/d81/l8a 0 2024-03-11T19:55:59.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/495: truncate dd/d13/d18/f75 1340966 0 2024-03-11T19:55:59.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/496: creat dd/d13/d32/f8b x:0 0 0 2024-03-11T19:55:59.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/525: dwrite d4/fa9 [0,4194304] 0 2024-03-11T19:55:59.497 INFO:tasks.workunit.client.0.smithi107.stdout:3/544: write d1/f6e [274120,80446] 0 2024-03-11T19:55:59.497 INFO:tasks.workunit.client.0.smithi107.stdout:3/545: chown d1/d10/d4e/d8c 107 1 2024-03-11T19:55:59.497 INFO:tasks.workunit.client.0.smithi107.stdout:3/546: chown d1/d10/d13/d4a/c87 1122 1 2024-03-11T19:55:59.498 INFO:tasks.workunit.client.0.smithi107.stdout:3/547: symlink d1/d10/d13/d6c/d2d/d4d/d91/la4 0 2024-03-11T19:55:59.512 INFO:tasks.workunit.client.0.smithi107.stdout:5/654: dwrite da/d14/d18/d1e/d9b/f57 [0,4194304] 0 2024-03-11T19:55:59.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/655: chown da/f85 221776653 1 2024-03-11T19:55:59.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/548: mkdir d1/d10/d13/d6c/d2d/d4d/d91/da5 0 2024-03-11T19:55:59.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/549: write d1/d10/d26/d17/f81 [169807,65040] 0 2024-03-11T19:55:59.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/656: rename da/d14/d43/d7f/d98 to da/d14/d38/d59/d5f/dc9 0 2024-03-11T19:55:59.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/657: symlink da/d14/d43/d47/lca 0 2024-03-11T19:55:59.532 INFO:tasks.workunit.client.0.smithi107.stdout:5/658: symlink da/d14/d43/lcb 0 2024-03-11T19:55:59.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/599: dwrite d6/da/d16/f2b [0,4194304] 0 2024-03-11T19:55:59.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/600: readlink d6/da/d16/d91/d97/d9b/laf 0 2024-03-11T19:55:59.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/601: readlink d6/da/d16/l5b 0 2024-03-11T19:55:59.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/602: mknod d6/da/d16/d5c/d7d/cbd 0 2024-03-11T19:55:59.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/603: unlink d6/da/d49/c7a 0 2024-03-11T19:55:59.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/604: creat d6/da/d16/d5c/d7d/d60/fbe x:0 0 0 2024-03-11T19:55:59.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/605: truncate d6/da/d16/d5c/d7d/d60/fbe 605754 0 2024-03-11T19:55:59.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/526: dwrite d4/d1d/d5c/d64/f84 [0,4194304] 0 2024-03-11T19:55:59.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/527: truncate d4/d1d/d1f/d28/f67 654921 0 2024-03-11T19:55:59.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/528: truncate d4/d1d/d1f/f39 1341086 0 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/659: dread da/d14/d38/d59/f64 [0,4194304] 0 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/660: creat da/fcc x:0 0 0 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/661: truncate da/d14/d18/d1e/d9b/fc0 347186 0 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/662: write da/d14/d18/d1e/d9b/fc0 [788794,55455] 0 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/663: chown da/d14/d18/d1e/d9b/db3 63 1 2024-03-11T19:55:59.567 INFO:tasks.workunit.client.0.smithi107.stdout:3/550: dwrite d1/d10/f57 [8388608,4194304] 0 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/664: mkdir da/d14/d18/d44/dcd 0 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/665: chown da/d14/d18/c55 266 1 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/551: mkdir d1/d10/d26/da6 0 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/606: write d6/f7e [424668,71168] 0 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/607: dread - d6/f7b zero size 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/608: chown d6/da/d16/d5c/f73 5100694 1 2024-03-11T19:55:59.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/552: mknod d1/d10/d13/d6c/ca7 0 2024-03-11T19:55:59.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/609: getdents d6/da/d16/d5c/d7d 0 2024-03-11T19:55:59.576 INFO:tasks.workunit.client.0.smithi107.stdout:7/610: truncate d6/d9/d4e/f59 3986781 0 2024-03-11T19:55:59.577 INFO:tasks.workunit.client.0.smithi107.stdout:9/529: dread d4/d1c/d37/f50 [0,4194304] 0 2024-03-11T19:55:59.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/611: link d6/da/fbc d6/da/d16/fbf 0 2024-03-11T19:55:59.579 INFO:tasks.workunit.client.0.smithi107.stdout:7/612: chown d6/da/d16/d5c/f70 129589504 1 2024-03-11T19:55:59.580 INFO:tasks.workunit.client.0.smithi107.stdout:9/530: symlink d4/d1d/lc3 0 2024-03-11T19:55:59.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/546: sync 2024-03-11T19:55:59.587 INFO:tasks.workunit.client.0.smithi107.stdout:1/593: sync 2024-03-11T19:55:59.587 INFO:tasks.workunit.client.0.smithi107.stdout:1/594: fdatasync d6/d28/f7c 0 2024-03-11T19:55:59.587 INFO:tasks.workunit.client.0.smithi107.stdout:4/547: creat d4/d7/d1f/d2b/d19/fa7 x:0 0 0 2024-03-11T19:55:59.588 INFO:tasks.workunit.client.0.smithi107.stdout:3/553: write d1/d10/d21/d43/d48/f69 [1671528,31104] 0 2024-03-11T19:55:59.589 INFO:tasks.workunit.client.0.smithi107.stdout:9/531: rename d4/d1c/d37/l95 to d4/d1d/lc4 0 2024-03-11T19:55:59.589 INFO:tasks.workunit.client.0.smithi107.stdout:9/532: creat d4/d1d/d1f/d78/fc5 x:0 0 0 2024-03-11T19:55:59.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/666: write da/d31/d42/f1f [2895440,116260] 0 2024-03-11T19:55:59.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/548: rename d4/d7/d2f/d33/f88 to d4/d7/d1f/d2b/d19/d75/d8e/fa8 0 2024-03-11T19:55:59.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/613: dwrite d6/da/d16/d5c/d7d/d1b/d76/d80/fb0 [0,4194304] 0 2024-03-11T19:55:59.630 INFO:tasks.workunit.client.0.smithi107.stdout:4/549: write d4/d7/d1f/d2b/f56 [2127623,115497] 0 2024-03-11T19:55:59.632 INFO:tasks.workunit.client.0.smithi107.stdout:4/550: link d4/d7/d1f/d2b/c86 d4/d7/d1f/d2b/d19/d9a/ca9 0 2024-03-11T19:55:59.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/667: dread da/d14/d38/d46/f9f [0,4194304] 0 2024-03-11T19:55:59.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/551: chown d4/d7/d2f/d3c/f73 24417 1 2024-03-11T19:55:59.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/554: dread d1/d10/d4e/d73/f20 [0,4194304] 0 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/555: getdents d1/d10/d4e/d73/d8d/d9c 0 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:4/552: truncate d4/d7/d1f/d2b/d24/d26/f84 3392380 0 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/556: creat d1/d10/d21/d43/d48/d98/fa8 x:0 0 0 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/557: write d1/d10/d21/d43/d48/d98/f9e [576938,66585] 0 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/558: chown d1/d10/d13/d4a/d4c/f7b 40 1 2024-03-11T19:55:59.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/559: creat d1/d10/d13/d6c/d5d/fa9 x:0 0 0 2024-03-11T19:55:59.639 INFO:tasks.workunit.client.0.smithi107.stdout:4/553: getdents d4/d7/d2f/d3b 0 2024-03-11T19:55:59.641 INFO:tasks.workunit.client.0.smithi107.stdout:3/560: symlink d1/d10/d13/d6c/d2d/d4d/d91/laa 0 2024-03-11T19:55:59.641 INFO:tasks.workunit.client.0.smithi107.stdout:3/561: chown d1/d10/d13/d4a/d4c/c7c 1381085 1 2024-03-11T19:55:59.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/595: dwrite d6/d7e/d81/f29 [0,4194304] 0 2024-03-11T19:55:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/562: link d1/d10/l5f d1/d10/d26/lab 0 2024-03-11T19:55:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:3/563: link d1/d10/d13/d6c/d5d/d76/l9b d1/d10/d4e/d73/d8d/lac 0 2024-03-11T19:55:59.664 INFO:tasks.workunit.client.0.smithi107.stdout:1/596: read d6/d7e/d81/d27/d2c/d32/dc1/f63 [704430,121954] 0 2024-03-11T19:55:59.665 INFO:tasks.workunit.client.0.smithi107.stdout:1/597: truncate d6/d7e/d81/d27/d2c/f30 3148546 0 2024-03-11T19:55:59.665 INFO:tasks.workunit.client.0.smithi107.stdout:1/598: fdatasync d6/d7e/d81/f24 0 2024-03-11T19:55:59.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/599: write d6/d19/fb5 [167204,4226] 0 2024-03-11T19:55:59.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/600: getdents d6/d7e/d81/d27/d2c/d32/dc1/d95 0 2024-03-11T19:55:59.669 INFO:tasks.workunit.client.0.smithi107.stdout:1/601: chown d6/d5b/l61 471 1 2024-03-11T19:55:59.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/564: write d1/d10/d21/d43/d48/f56 [287145,123909] 0 2024-03-11T19:55:59.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/565: readlink d1/d10/l62 0 2024-03-11T19:55:59.676 INFO:tasks.workunit.client.0.smithi107.stdout:3/566: truncate d1/d10/d21/d43/d48/f69 918477 0 2024-03-11T19:55:59.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/614: dwrite d6/d9/d4e/f59 [0,4194304] 0 2024-03-11T19:55:59.689 INFO:tasks.workunit.client.0.smithi107.stdout:3/567: dread d1/d10/d13/d4a/f93 [0,4194304] 0 2024-03-11T19:55:59.689 INFO:tasks.workunit.client.0.smithi107.stdout:3/568: dread - d1/d10/d13/d6c/d5d/f94 zero size 2024-03-11T19:55:59.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/668: dwrite da/d31/f4f [0,4194304] 0 2024-03-11T19:55:59.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/602: dread d6/d7e/d81/d27/d2c/d72/f8a [0,4194304] 0 2024-03-11T19:55:59.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/603: chown d6/d7e/d81/d27/d2c/d7d 41568731 1 2024-03-11T19:55:59.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/604: mknod d6/d7e/da4/cc2 0 2024-03-11T19:55:59.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/554: dwrite d4/d7/d2f/d33/f66 [0,4194304] 0 2024-03-11T19:55:59.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/555: truncate d4/d7/d1f/d2b/f5d 1573044 0 2024-03-11T19:55:59.706 INFO:tasks.workunit.client.0.smithi107.stdout:7/615: dread d6/da/d16/f2b [0,4194304] 0 2024-03-11T19:55:59.706 INFO:tasks.workunit.client.0.smithi107.stdout:7/616: write d6/da/d16/d5c/d7d/d60/fba [549296,123860] 0 2024-03-11T19:55:59.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/605: dread d6/d28/f41 [0,4194304] 0 2024-03-11T19:55:59.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/556: creat d4/d7/d2f/d33/d3a/d79/faa x:0 0 0 2024-03-11T19:55:59.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/606: mkdir d6/d7e/d81/d27/d2c/d72/dc3 0 2024-03-11T19:55:59.720 INFO:tasks.workunit.client.0.smithi107.stdout:4/557: rename d4/d7/f98 to d4/fab 0 2024-03-11T19:55:59.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/558: rmdir d4/d7/d2f/d33/d6d 0 2024-03-11T19:55:59.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/617: write d6/d9/d57/d65/f74 [3019773,88652] 0 2024-03-11T19:55:59.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/618: dread - d6/da/d16/d5c/d7d/fa9 zero size 2024-03-11T19:55:59.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/619: mknod d6/d9/cc0 0 2024-03-11T19:55:59.731 INFO:tasks.workunit.client.0.smithi107.stdout:7/620: symlink d6/da/d16/d91/lc1 0 2024-03-11T19:55:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/559: rmdir d4/d7 39 2024-03-11T19:55:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/621: mknod d6/d21/dab/cc2 0 2024-03-11T19:55:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/622: fsync d6/d9/f85 0 2024-03-11T19:55:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/623: chown d6/d21 219651 1 2024-03-11T19:55:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/624: fsync d6/d9/d57/f9e 0 2024-03-11T19:55:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/625: creat d6/da/d16/d5c/d7d/d60/fc3 x:0 0 0 2024-03-11T19:55:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:5/669: dread da/f85 [0,4194304] 0 2024-03-11T19:55:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/626: creat d6/da/d16/d5c/d7d/d60/d61/d78/fc4 x:0 0 0 2024-03-11T19:55:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/627: readlink d6/da/l8e 0 2024-03-11T19:55:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/628: chown d6/da/d16/d5c/d7d/c28 953 1 2024-03-11T19:55:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/629: chown d6/d9/f6d 6375 1 2024-03-11T19:55:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/630: chown d6/d9/d57/l9f 9780065 1 2024-03-11T19:55:59.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/670: creat da/d14/d43/fce x:0 0 0 2024-03-11T19:55:59.744 INFO:tasks.workunit.client.0.smithi107.stdout:6/475: sync 2024-03-11T19:55:59.747 INFO:tasks.workunit.client.0.smithi107.stdout:5/671: rmdir da/d14/d18/d1e/d9b/d73 39 2024-03-11T19:55:59.750 INFO:tasks.workunit.client.0.smithi107.stdout:3/569: dwrite d1/d10/d13/f61 [0,4194304] 0 2024-03-11T19:55:59.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/672: symlink da/d14/d18/d1e/d9b/d72/lcf 0 2024-03-11T19:55:59.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/570: symlink d1/d10/d26/da6/lad 0 2024-03-11T19:55:59.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/673: mkdir da/d14/d38/dd0 0 2024-03-11T19:55:59.753 INFO:tasks.workunit.client.0.smithi107.stdout:3/571: rename d1/d10/d13/d6c/d5d/fa9 to d1/d10/d26/d17/fae 0 2024-03-11T19:55:59.753 INFO:tasks.workunit.client.0.smithi107.stdout:4/560: dread d4/d7/d1f/d2b/d24/d26/d29/f32 [0,4194304] 0 2024-03-11T19:55:59.754 INFO:tasks.workunit.client.0.smithi107.stdout:3/572: symlink d1/d10/d13/d6c/laf 0 2024-03-11T19:55:59.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/561: creat d4/d7/d2f/d3b/fac x:0 0 0 2024-03-11T19:55:59.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/573: mkdir d1/d10/d21/d43/d48/db0 0 2024-03-11T19:55:59.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/574: link d1/d10/d13/d6c/l24 d1/d10/d2a/d7e/lb1 0 2024-03-11T19:55:59.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/562: truncate d4/d7/d1f/d2b/f56 894679 0 2024-03-11T19:55:59.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/563: write f3 [5042112,52695] 0 2024-03-11T19:55:59.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/607: dwrite d6/d7e/d81/d27/d2c/d32/f7f [0,4194304] 0 2024-03-11T19:55:59.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/564: link d4/d7/d2f/d33/d3a/d79/c94 d4/d7/d1f/d2b/d19/d75/cad 0 2024-03-11T19:55:59.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/631: dwrite d6/da/f8a [0,4194304] 0 2024-03-11T19:55:59.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/632: chown d6/da/d16/d5c/f4a 1 1 2024-03-11T19:55:59.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/633: mkdir d6/da/d16/d5c/d7d/dc5 0 2024-03-11T19:55:59.802 INFO:tasks.workunit.client.0.smithi107.stdout:6/476: dwrite d10/d19/f31 [4194304,4194304] 0 2024-03-11T19:55:59.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/477: dread - d10/d3d/d54/f80 zero size 2024-03-11T19:55:59.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/608: write d6/d7e/d81/f29 [1759358,50210] 0 2024-03-11T19:55:59.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/609: chown d6/d7e/da5/dac/d62/da3/fba 397413322 1 2024-03-11T19:55:59.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/478: mknod d10/d3d/d54/ca3 0 2024-03-11T19:55:59.807 INFO:tasks.workunit.client.0.smithi107.stdout:5/674: dwrite f4 [8388608,4194304] 0 2024-03-11T19:55:59.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/675: mknod da/d14/d43/cd1 0 2024-03-11T19:55:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:1/610: rename d6/d7e/d88 to d6/d28/dc4 0 2024-03-11T19:55:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/553: sync 2024-03-11T19:55:59.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/676: rename da/d14/d18/l74 to da/d14/d43/ld2 0 2024-03-11T19:55:59.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/677: fdatasync da/d14/d18/f4c 0 2024-03-11T19:55:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/554: symlink d1/d38/d75/d76/lba 0 2024-03-11T19:55:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:5/678: truncate da/d14/d38/d59/f64 1733531 0 2024-03-11T19:55:59.837 INFO:tasks.workunit.client.0.smithi107.stdout:5/679: dread da/d14/d18/f51 [0,4194304] 0 2024-03-11T19:55:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/575: dwrite d1/d10/d4e/d73/f41 [0,4194304] 0 2024-03-11T19:55:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/634: dwrite d6/da/d16/d5c/d7d/d60/d61/d78/fc4 [0,4194304] 0 2024-03-11T19:55:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/576: fsync d1/d10/d2a/d7e/d8b/f97 0 2024-03-11T19:55:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/577: fdatasync d1/d10/d13/d4a/d4c/f2f 0 2024-03-11T19:55:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/635: write d6/f7e [3665116,25819] 0 2024-03-11T19:55:59.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/578: rename d1/d10/d2a/c52 to d1/d10/d13/d6c/d2d/d4d/cb2 0 2024-03-11T19:55:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/579: write d1/d10/d13/d6c/d2d/d4d/f71 [2670465,94896] 0 2024-03-11T19:55:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/580: read - d1/d10/d13/d6c/d5d/f94 zero size 2024-03-11T19:55:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/581: stat d1/d10/d26/da6 0 2024-03-11T19:55:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/582: truncate d1/d10/d13/d4a/f93 693574 0 2024-03-11T19:55:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:7/636: creat d6/d9/d57/d65/fc6 x:0 0 0 2024-03-11T19:55:59.844 INFO:tasks.workunit.client.0.smithi107.stdout:3/583: mknod d1/d10/d13/d6c/d2d/d4d/cb3 0 2024-03-11T19:55:59.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/584: mkdir d1/d10/d13/d6c/d2d/d4d/d91/db4 0 2024-03-11T19:55:59.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/585: chown d1/d10/d26 3 1 2024-03-11T19:55:59.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/611: dwrite d6/d28/f41 [0,4194304] 0 2024-03-11T19:55:59.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/479: dwrite d10/d3d/f98 [0,4194304] 0 2024-03-11T19:55:59.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/480: write d10/d19/d3b/f8b [1002508,23939] 0 2024-03-11T19:55:59.871 INFO:tasks.workunit.client.0.smithi107.stdout:3/586: write d1/d10/d13/d6c/f32 [3897338,105831] 0 2024-03-11T19:55:59.873 INFO:tasks.workunit.client.0.smithi107.stdout:3/587: creat d1/d10/d13/d4a/d4c/d49/fb5 x:0 0 0 2024-03-11T19:55:59.881 INFO:tasks.workunit.client.0.smithi107.stdout:3/588: dread d1/d10/d26/d17/f81 [0,4194304] 0 2024-03-11T19:55:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/589: mknod d1/d10/d21/d43/d82/cb6 0 2024-03-11T19:55:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:3/590: write d1/d10/d4e/fa2 [852423,30047] 0 2024-03-11T19:55:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/591: write d1/d10/d13/f2c [1402812,124428] 0 2024-03-11T19:55:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/592: fdatasync d1/d10/d26/f25 0 2024-03-11T19:55:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/593: dread - d1/d10/d21/d43/d4f/f59 zero size 2024-03-11T19:55:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/594: unlink d1/d10/d13/d6c/d5d/c66 0 2024-03-11T19:55:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:3/595: truncate d1/d10/d21/d43/f84 1169411 0 2024-03-11T19:55:59.884 INFO:tasks.workunit.client.0.smithi107.stdout:3/596: mknod d1/d10/d4e/d73/d8d/d9c/cb7 0 2024-03-11T19:55:59.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/637: dwrite d6/da/d16/d5c/d7d/d1b/d76/d80/fb0 [0,4194304] 0 2024-03-11T19:55:59.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/638: fdatasync d6/da/d16/f36 0 2024-03-11T19:55:59.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/639: chown d6/f5f 790450207 1 2024-03-11T19:55:59.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/640: creat d6/d9/d57/d65/fc7 x:0 0 0 2024-03-11T19:55:59.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/641: rename d6/da/d16/d5c/d7d/d1b/d76 to d6/da/d16/d91/dc8 0 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/597: dwrite d1/f5 [4194304,4194304] 0 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/598: stat d1/d10/d21/d43 0 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/599: chown d1/d10/d13/f2c 27562 1 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:7/642: mknod d6/da/d16/d5c/d7d/d60/d61/d9c/cc9 0 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/481: dwrite d10/d19/d3b/f99 [0,4194304] 0 2024-03-11T19:55:59.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/482: rename d10/d3d/d54 to d10/d3d/d54/da4 22 2024-03-11T19:55:59.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/483: stat cf 0 2024-03-11T19:55:59.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/600: creat d1/d10/d13/d6c/d2d/d4d/d91/db4/fb8 x:0 0 0 2024-03-11T19:55:59.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/484: mkdir d10/d16/d2b/da5 0 2024-03-11T19:55:59.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/485: creat d10/d19/fa6 x:0 0 0 2024-03-11T19:55:59.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/486: readlink lb 0 2024-03-11T19:55:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:7/643: creat d6/d9/d57/da2/fca x:0 0 0 2024-03-11T19:55:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/612: dwrite d6/d28/fb2 [0,4194304] 0 2024-03-11T19:55:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:1/613: mknod d6/d86/dc0/cc5 0 2024-03-11T19:55:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/601: mknod d1/d45/d85/cb9 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/487: link d10/d16/d1f/d51/d76/f93 d10/d16/d1f/d51/d76/d7f/fa7 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/488: chown d10/d3d/d54 0 1 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/602: write d1/d10/d4e/fa2 [1012837,30618] 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/603: truncate d1/d10/d21/d43/d48/f9f 218967 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/489: creat d10/d16/d24/d2e/d9d/fa8 x:0 0 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/490: stat d10/d16/d2b/f9a 0 2024-03-11T19:55:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/491: chown d10/d16/d2b/d8d 79 1 2024-03-11T19:55:59.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/604: symlink d1/d45/lba 0 2024-03-11T19:55:59.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/605: dread - d1/d10/d13/d4a/d4c/d49/f54 zero size 2024-03-11T19:55:59.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/606: mkdir d1/dbb 0 2024-03-11T19:55:59.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/607: chown d1/d10/d13 0 1 2024-03-11T19:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/608: rmdir d1/d10/d21 39 2024-03-11T19:55:59.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/614: rename d6/d7e/d81/d27/d2c/f79 to d6/d28/dc4/fc6 0 2024-03-11T19:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout:1/615: mkdir d6/d7e/da5/dc7 0 2024-03-11T19:55:59.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/644: dwrite d6/da/d16/d5c/d7d/d60/d61/fb3 [0,4194304] 0 2024-03-11T19:55:59.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/645: creat d6/d21/fcb x:0 0 0 2024-03-11T19:55:59.962 INFO:tasks.workunit.client.0.smithi107.stdout:7/646: fdatasync d6/da/d16/f6f 0 2024-03-11T19:55:59.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/647: truncate d6/d9/d57/d65/f90 245941 0 2024-03-11T19:55:59.964 INFO:tasks.workunit.client.0.smithi107.stdout:7/648: readlink d6/da/d16/d5c/d7d/d60/lb6 0 2024-03-11T19:55:59.968 INFO:tasks.workunit.client.0.smithi107.stdout:9/533: sync 2024-03-11T19:55:59.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/497: sync 2024-03-11T19:55:59.968 INFO:tasks.workunit.client.0.smithi107.stdout:9/534: creat d4/d9/fc6 x:0 0 0 2024-03-11T19:55:59.969 INFO:tasks.workunit.client.0.smithi107.stdout:9/535: fsync d4/d1d/d1f/d78/fc5 0 2024-03-11T19:55:59.969 INFO:tasks.workunit.client.0.smithi107.stdout:0/589: sync 2024-03-11T19:55:59.969 INFO:tasks.workunit.client.0.smithi107.stdout:0/590: chown de/d11/d23/d32/d34/d39/l52 57943 1 2024-03-11T19:55:59.969 INFO:tasks.workunit.client.0.smithi107.stdout:3/609: dwrite d1/d10/d4e/d73/f20 [4194304,4194304] 0 2024-03-11T19:55:59.969 INFO:tasks.workunit.client.0.smithi107.stdout:3/610: chown d1/d10/d4e/d73/d8d/lac 0 1 2024-03-11T19:55:59.970 INFO:tasks.workunit.client.0.smithi107.stdout:3/611: getdents d1/d10/d21/d43/d48/db0 0 2024-03-11T19:55:59.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/498: symlink dd/de/d49/d7e/l8c 0 2024-03-11T19:55:59.970 INFO:tasks.workunit.client.0.smithi107.stdout:0/591: creat de/d11/d23/d32/d34/d39/d3a/d72/fb2 x:0 0 0 2024-03-11T19:55:59.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/499: creat dd/d13/d18/d81/f8d x:0 0 0 2024-03-11T19:55:59.973 INFO:tasks.workunit.client.0.smithi107.stdout:8/500: creat dd/d16/f8e x:0 0 0 2024-03-11T19:55:59.974 INFO:tasks.workunit.client.0.smithi107.stdout:9/536: rename d4/d9/fc6 to d4/d1d/d5c/fc7 0 2024-03-11T19:55:59.975 INFO:tasks.workunit.client.0.smithi107.stdout:6/492: dwrite d10/d16/d1f/f28 [0,4194304] 0 2024-03-11T19:55:59.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/493: stat d10/d19/d38/d40/l4f 0 2024-03-11T19:55:59.978 INFO:tasks.workunit.client.0.smithi107.stdout:3/612: rename d1/d10/d13/d4a/d4c/c18 to d1/d10/d21/cbc 0 2024-03-11T19:55:59.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/649: dread d6/da/fbb [0,4194304] 0 2024-03-11T19:55:59.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/494: write d10/d16/d1f/f79 [2740013,62707] 0 2024-03-11T19:55:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/495: fsync d10/d16/d2b/d39/d6a/f83 0 2024-03-11T19:55:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/496: fdatasync d10/d16/d1f/f28 0 2024-03-11T19:55:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/497: chown d10/d3d/d54/ca3 80806 1 2024-03-11T19:55:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/498: chown d10/d16/d1f/d51/f87 910034 1 2024-03-11T19:55:59.998 INFO:tasks.workunit.client.0.smithi107.stdout:3/613: dread d1/d10/d13/d6c/f32 [0,4194304] 0 2024-03-11T19:56:00.000 INFO:tasks.workunit.client.0.smithi107.stdout:3/614: link d1/d10/d13/d6c/d2d/d4d/cb2 d1/d10/d4e/d73/cbd 0 2024-03-11T19:56:00.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/650: dread d6/da/fa3 [0,4194304] 0 2024-03-11T19:56:00.013 INFO:tasks.workunit.client.0.smithi107.stdout:7/651: rename d6/da/d4d/f96 to d6/d9/d57/da2/fcc 0 2024-03-11T19:56:00.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/615: dread d1/d10/f6f [0,4194304] 0 2024-03-11T19:56:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/652: symlink d6/da/d16/d91/d97/lcd 0 2024-03-11T19:56:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/653: chown d6 17140826 1 2024-03-11T19:56:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/654: symlink d6/da/d16/d5c/d7d/dc5/lce 0 2024-03-11T19:56:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/655: unlink d6/ff 0 2024-03-11T19:56:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/656: write d6/da/fbb [5061533,6438] 0 2024-03-11T19:56:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/657: fsync d6/da/f45 0 2024-03-11T19:56:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:7/658: chown d6/d9/d4e 360 1 2024-03-11T19:56:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:7/659: mknod d6/d9/d4e/ccf 0 2024-03-11T19:56:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/660: symlink d6/da/d16/d91/d97/ld0 0 2024-03-11T19:56:00.023 INFO:tasks.workunit.client.0.smithi107.stdout:8/501: dwrite dd/d13/d32/f8b [0,4194304] 0 2024-03-11T19:56:00.023 INFO:tasks.workunit.client.0.smithi107.stdout:7/661: link d6/d21/c75 d6/cd1 0 2024-03-11T19:56:00.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/537: dwrite d4/d9/dae/d6d/db0/fc2 [0,4194304] 0 2024-03-11T19:56:00.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/538: dread - d4/d1d/d1f/d78/fc0 zero size 2024-03-11T19:56:00.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/502: creat dd/de/d1b/d20/d56/d87/f8f x:0 0 0 2024-03-11T19:56:00.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/503: chown dd/de/d1b/d20/d5e/c7c 21529 1 2024-03-11T19:56:00.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/504: write dd/de/d1b/d20/f6a [516553,8725] 0 2024-03-11T19:56:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:8/505: mkdir dd/de/d1b/d2c/d90 0 2024-03-11T19:56:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/539: readlink d4/d1d/d1f/d28/d2c/d96/la0 0 2024-03-11T19:56:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/540: dread - d4/d9/dae/d6d/f68 zero size 2024-03-11T19:56:00.033 INFO:tasks.workunit.client.0.smithi107.stdout:8/506: getdents dd/de/d49 0 2024-03-11T19:56:00.033 INFO:tasks.workunit.client.0.smithi107.stdout:8/507: fdatasync dd/d13/d32/f8b 0 2024-03-11T19:56:00.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/499: dwrite d10/d19/f65 [0,4194304] 0 2024-03-11T19:56:00.038 INFO:tasks.workunit.client.0.smithi107.stdout:8/508: unlink dd/d13/d32/d51/l71 0 2024-03-11T19:56:00.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/616: write d1/d45/f75 [438931,96365] 0 2024-03-11T19:56:00.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/617: getdents d1/d10/d26 0 2024-03-11T19:56:00.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/662: dread d6/d9/fa6 [0,4194304] 0 2024-03-11T19:56:00.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/618: dread d1/d10/f6f [0,4194304] 0 2024-03-11T19:56:00.056 INFO:tasks.workunit.client.0.smithi107.stdout:3/619: mknod d1/d45/d85/cbe 0 2024-03-11T19:56:00.056 INFO:tasks.workunit.client.0.smithi107.stdout:3/620: write d1/d10/d26/d17/f81 [274934,120972] 0 2024-03-11T19:56:00.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/509: write dd/de/d49/f73 [1436057,87882] 0 2024-03-11T19:56:00.061 INFO:tasks.workunit.client.0.smithi107.stdout:8/510: readlink dd/de/d1b/d2c/l3e 0 2024-03-11T19:56:00.062 INFO:tasks.workunit.client.0.smithi107.stdout:4/565: sync 2024-03-11T19:56:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:7/663: dread d6/da/d16/d5c/d7d/d1b/f3c [0,4194304] 0 2024-03-11T19:56:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:7/664: chown d6/d21/c82 245898 1 2024-03-11T19:56:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:7/665: dread - d6/da/d16/d5c/d7d/fa9 zero size 2024-03-11T19:56:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:7/666: chown d6/d9/d4e/l50 2 1 2024-03-11T19:56:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:7/667: stat d6/cd1 0 2024-03-11T19:56:00.079 INFO:tasks.workunit.client.0.smithi107.stdout:7/668: mkdir d6/da/d16/d5c/d7d/d60/dd2 0 2024-03-11T19:56:00.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/669: symlink d6/da/d16/d5c/d7d/d60/d61/ld3 0 2024-03-11T19:56:00.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/670: getdents d6/da 0 2024-03-11T19:56:00.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/500: dwrite d10/d19/f31 [4194304,4194304] 0 2024-03-11T19:56:00.086 INFO:tasks.workunit.client.0.smithi107.stdout:7/671: creat d6/fd4 x:0 0 0 2024-03-11T19:56:00.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/511: read dd/de/d1b/d20/f43 [1540208,97868] 0 2024-03-11T19:56:00.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/672: creat d6/da/d4d/fd5 x:0 0 0 2024-03-11T19:56:00.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/673: write d6/fd4 [40741,65665] 0 2024-03-11T19:56:00.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/674: creat d6/da/d4d/fd6 x:0 0 0 2024-03-11T19:56:00.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/675: write d6/f7e [2648303,88755] 0 2024-03-11T19:56:00.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/541: dwrite d4/d1d/d5c/d64/fa8 [0,4194304] 0 2024-03-11T19:56:00.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/512: truncate dd/d13/d32/f8b 4109186 0 2024-03-11T19:56:00.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/513: chown dd/de/d49/f5b 29072 1 2024-03-11T19:56:00.097 INFO:tasks.workunit.client.0.smithi107.stdout:8/514: chown dd/d13/d18/f37 131011 1 2024-03-11T19:56:00.097 INFO:tasks.workunit.client.0.smithi107.stdout:7/676: rename d6/da/d16/d5c/d7d/cbd to d6/d9/dae/cd7 0 2024-03-11T19:56:00.105 INFO:tasks.workunit.client.0.smithi107.stdout:7/677: symlink d6/da/d49/ld8 0 2024-03-11T19:56:00.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/566: dread d4/d7/d2f/d3b/f58 [0,4194304] 0 2024-03-11T19:56:00.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/567: truncate d4/d7/d1f/d2b/d24/d26/d29/d36/f57 103038 0 2024-03-11T19:56:00.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/542: write d4/d1d/f3b [84468,30714] 0 2024-03-11T19:56:00.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/568: creat d4/d7/d1f/d2b/d24/d26/fae x:0 0 0 2024-03-11T19:56:00.132 INFO:tasks.workunit.client.0.smithi107.stdout:6/501: write d10/f13 [1099350,3462] 0 2024-03-11T19:56:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:6/502: creat d10/d19/d38/d60/fa9 x:0 0 0 2024-03-11T19:56:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:4/569: symlink d4/d7/da0/laf 0 2024-03-11T19:56:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/543: rmdir d4/d9/d9f 39 2024-03-11T19:56:00.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/678: dread d6/da/d16/d5c/d7d/d1b/f1c [0,4194304] 0 2024-03-11T19:56:00.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/679: fdatasync d6/d9/f8 0 2024-03-11T19:56:00.134 INFO:tasks.workunit.client.0.smithi107.stdout:3/621: dwrite d1/d10/d13/f61 [0,4194304] 0 2024-03-11T19:56:00.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/544: creat d4/d1d/d5c/d64/db3/fc8 x:0 0 0 2024-03-11T19:56:00.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/545: chown d4/d1d/d1f/d28/c57 297751054 1 2024-03-11T19:56:00.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/680: rename d6/da/d16/d5c/d7d/d60/d61/d78/fc4 to d6/d9/fd9 0 2024-03-11T19:56:00.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/570: symlink d4/d7/d1f/d2b/d24/d26/lb0 0 2024-03-11T19:56:00.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/546: rename d4/d1d/d1f/d78/cb1 to d4/d1c/cc9 0 2024-03-11T19:56:00.145 INFO:tasks.workunit.client.0.smithi107.stdout:7/681: creat d6/d9/d57/fda x:0 0 0 2024-03-11T19:56:00.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/555: sync 2024-03-11T19:56:00.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/556: stat d1/d1d/c1f 0 2024-03-11T19:56:00.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/557: chown d1/d1a/d34/d4f/c69 58 1 2024-03-11T19:56:00.149 INFO:tasks.workunit.client.0.smithi107.stdout:2/558: unlink d1/d1a/d1e/d26/f32 0 2024-03-11T19:56:00.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/559: write d1/d38/d75/fa0 [799741,128255] 0 2024-03-11T19:56:00.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/560: rmdir d1/d38/d75 39 2024-03-11T19:56:00.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/561: getdents d1/d21 0 2024-03-11T19:56:00.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/562: getdents d1/d1a/d1e/d26 0 2024-03-11T19:56:00.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/563: mkdir d1/d21/dbb 0 2024-03-11T19:56:00.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/564: truncate d1/d1a/fa3 863900 0 2024-03-11T19:56:00.163 INFO:tasks.workunit.client.0.smithi107.stdout:2/565: mknod d1/d1d/cbc 0 2024-03-11T19:56:00.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/566: dread d1/d21/fa7 [0,4194304] 0 2024-03-11T19:56:00.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/567: creat d1/d1a/d34/d4f/d63/db6/fbd x:0 0 0 2024-03-11T19:56:00.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/568: stat d1/d21/f28 0 2024-03-11T19:56:00.174 INFO:tasks.workunit.client.0.smithi107.stdout:4/571: dwrite d4/fab [4194304,4194304] 0 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/572: rename d4/d7/d1f/d2b/d24/d26/d29/d36/f7d to d4/d7/d1f/d2b/fb1 0 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/573: dread - d4/d7/d2f/d3c/f96 zero size 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/503: dwrite d10/d16/d24/f58 [0,4194304] 0 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/504: fsync d10/d16/d24/f5b 0 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/505: chown d10/d16/d1f/d51/d7a 455071 1 2024-03-11T19:56:00.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/506: write d10/f53 [181570,115661] 0 2024-03-11T19:56:00.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/574: creat d4/d7/d1f/d2b/d19/d75/fb2 x:0 0 0 2024-03-11T19:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout:6/507: truncate d10/d16/d24/f68 3838789 0 2024-03-11T19:56:00.183 INFO:tasks.workunit.client.0.smithi107.stdout:6/508: truncate d10/d16/d24/f5b 176755 0 2024-03-11T19:56:00.186 INFO:tasks.workunit.client.0.smithi107.stdout:6/509: unlink d10/d3d/f4c 0 2024-03-11T19:56:00.188 INFO:tasks.workunit.client.0.smithi107.stdout:6/510: creat d10/d16/d2b/da5/faa x:0 0 0 2024-03-11T19:56:00.188 INFO:tasks.workunit.client.0.smithi107.stdout:6/511: fdatasync d10/d16/d24/d2e/f34 0 2024-03-11T19:56:00.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/512: rmdir d10/d19 39 2024-03-11T19:56:00.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/622: dwrite d1/d10/d13/d6c/d5d/f94 [0,4194304] 0 2024-03-11T19:56:00.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/623: truncate d1/d10/d13/d4a/d4c/d49/fb5 786334 0 2024-03-11T19:56:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/513: unlink d10/d16/d2b/d39/c82 0 2024-03-11T19:56:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:9/547: dwrite d4/d1d/d1f/f39 [0,4194304] 0 2024-03-11T19:56:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:3/624: creat d1/d45/fbf x:0 0 0 2024-03-11T19:56:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:3/625: rename d1/d10/d13/d6c/d5d/d76/l9b to d1/d10/d13/d6c/d2d/d4d/d91/db4/lc0 0 2024-03-11T19:56:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/626: readlink d1/lc 0 2024-03-11T19:56:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/627: link d1/d10/d21/d43/d48/f9f d1/d10/d4e/d73/d8d/fc1 0 2024-03-11T19:56:00.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/628: symlink d1/d10/d26/lc2 0 2024-03-11T19:56:00.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/629: rename d1/d10/d21/d43/d4f/f59 to d1/d10/d4e/d73/fc3 0 2024-03-11T19:56:00.211 INFO:tasks.workunit.client.0.smithi107.stdout:3/630: mknod d1/cc4 0 2024-03-11T19:56:00.222 INFO:tasks.workunit.client.0.smithi107.stdout:4/575: dwrite d4/d7/d1f/d2b/d24/d26/d29/d36/f8c [0,4194304] 0 2024-03-11T19:56:00.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/576: creat d4/d7/da0/fb3 x:0 0 0 2024-03-11T19:56:00.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/577: creat d4/d7/da0/fb4 x:0 0 0 2024-03-11T19:56:00.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/569: read d1/d1a/d34/d4f/f83 [149228,33472] 0 2024-03-11T19:56:00.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/578: getdents d4/d7/da0 0 2024-03-11T19:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout:7/682: dwrite d6/da/f8f [0,4194304] 0 2024-03-11T19:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout:7/683: write d6/da/d16/f6f [148699,20976] 0 2024-03-11T19:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/570: symlink d1/d1a/d34/d4f/d63/daa/lbe 0 2024-03-11T19:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/514: dwrite d10/d16/d24/d2e/f2f [0,4194304] 0 2024-03-11T19:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/515: write d10/d3d/f62 [14395,48596] 0 2024-03-11T19:56:00.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/548: dwrite d4/d9/dae/d6d/f2d [0,4194304] 0 2024-03-11T19:56:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:4/579: rename d4/d7/d1f/f20 to d4/d7/d1f/d2b/d19/d75/fb5 0 2024-03-11T19:56:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:7/684: creat d6/da/fdb x:0 0 0 2024-03-11T19:56:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:0/592: sync 2024-03-11T19:56:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:8/515: sync 2024-03-11T19:56:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/680: sync 2024-03-11T19:56:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/616: sync 2024-03-11T19:56:00.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/617: write d6/d7e/d81/d27/d2c/d32/f7f [4480623,2405] 0 2024-03-11T19:56:00.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/593: creat de/d11/d23/d24/da4/fb3 x:0 0 0 2024-03-11T19:56:00.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/594: chown de/d11/d23/d32/d34/d39/d55 883838 1 2024-03-11T19:56:00.254 INFO:tasks.workunit.client.0.smithi107.stdout:7/685: symlink d6/ldc 0 2024-03-11T19:56:00.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/516: mknod dd/de/d1b/d2c/d90/c91 0 2024-03-11T19:56:00.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/681: mknod da/d14/d18/d44/da5/cd3 0 2024-03-11T19:56:00.256 INFO:tasks.workunit.client.0.smithi107.stdout:1/618: truncate d6/d7e/d81/d27/f7b 654971 0 2024-03-11T19:56:00.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/682: symlink da/d14/d18/ld4 0 2024-03-11T19:56:00.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/683: stat da/d14/d38/d59/d5f/dc9 0 2024-03-11T19:56:00.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/684: chown f5 4442 1 2024-03-11T19:56:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:7/686: dread d6/da/d16/f98 [0,4194304] 0 2024-03-11T19:56:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/571: dread d1/f10 [0,4194304] 0 2024-03-11T19:56:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:3/631: dwrite d1/d10/d4e/d73/f20 [4194304,4194304] 0 2024-03-11T19:56:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/572: rmdir d1/d1d/d64/d97 39 2024-03-11T19:56:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/573: fsync d1/d38/d75/f7d 0 2024-03-11T19:56:00.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/632: symlink d1/d10/d13/d6c/d5d/d76/lc5 0 2024-03-11T19:56:00.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/633: write d1/d10/d21/d43/d48/f69 [1872118,52783] 0 2024-03-11T19:56:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/574: symlink d1/d21/d35/d67/lbf 0 2024-03-11T19:56:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/575: chown d1/d1a/d1e/c6d 0 1 2024-03-11T19:56:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/634: stat d1/d10/l1c 0 2024-03-11T19:56:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/635: truncate d1/d10/d13/d6c/d2d/d4d/d91/db4/fb8 477238 0 2024-03-11T19:56:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/636: creat d1/d10/d2a/d7e/d8b/fc6 x:0 0 0 2024-03-11T19:56:00.316 INFO:tasks.workunit.client.0.smithi107.stdout:7/687: dwrite d6/da/d4d/f71 [0,4194304] 0 2024-03-11T19:56:00.318 INFO:tasks.workunit.client.0.smithi107.stdout:1/619: dwrite d6/d7e/da5/dac/d82/f9a [0,4194304] 0 2024-03-11T19:56:00.318 INFO:tasks.workunit.client.0.smithi107.stdout:1/620: getdents d6/d7e/d81/d27/d2c/d32/dc1/d95 0 2024-03-11T19:56:00.318 INFO:tasks.workunit.client.0.smithi107.stdout:1/621: write d6/d7e/f8b [343280,131036] 0 2024-03-11T19:56:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:1/622: creat d6/d7e/d81/d27/d2c/d32/d58/fc8 x:0 0 0 2024-03-11T19:56:00.322 INFO:tasks.workunit.client.0.smithi107.stdout:6/516: write d10/d19/d3b/f64 [334914,60495] 0 2024-03-11T19:56:00.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/580: dwrite d4/d7/d1f/d2b/d24/d26/d29/d36/f8c [4194304,4194304] 0 2024-03-11T19:56:00.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/517: creat d10/d16/fab x:0 0 0 2024-03-11T19:56:00.325 INFO:tasks.workunit.client.0.smithi107.stdout:4/581: rename d4/d7/d2f/d33/l3d to d4/d7/d2f/d33/d7b/lb6 0 2024-03-11T19:56:00.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/582: creat d4/d7/d1f/d2b/d19/fb7 x:0 0 0 2024-03-11T19:56:00.327 INFO:tasks.workunit.client.0.smithi107.stdout:6/518: link d10/d19/d38/f5c d10/fac 0 2024-03-11T19:56:00.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/583: rmdir d4/d7/db 39 2024-03-11T19:56:00.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/584: mkdir d4/d7/db/db8 0 2024-03-11T19:56:00.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/595: dwrite de/d11/d23/d24/d4f/f78 [0,4194304] 0 2024-03-11T19:56:00.345 INFO:tasks.workunit.client.0.smithi107.stdout:7/688: write d6/da/f8a [2600900,104163] 0 2024-03-11T19:56:00.345 INFO:tasks.workunit.client.0.smithi107.stdout:8/517: dwrite dd/de/d1b/d20/f43 [0,4194304] 0 2024-03-11T19:56:00.345 INFO:tasks.workunit.client.0.smithi107.stdout:8/518: getdents dd/de 0 2024-03-11T19:56:00.346 INFO:tasks.workunit.client.0.smithi107.stdout:7/689: mkdir d6/da/d16/d91/dc8/d80/ddd 0 2024-03-11T19:56:00.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/519: mkdir dd/de/d49/d7e/d92 0 2024-03-11T19:56:00.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/690: link d6/da/d16/f2e d6/da/d16/d5c/d7d/d60/fde 0 2024-03-11T19:56:00.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/585: dwrite d4/d7/da0/fb3 [0,4194304] 0 2024-03-11T19:56:00.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/637: dwrite d1/d10/d13/d6c/d2d/d4d/d91/db4/fb8 [0,4194304] 0 2024-03-11T19:56:00.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/638: fdatasync d1/d10/d13/d6c/d2d/d4d/f53 0 2024-03-11T19:56:00.400 INFO:tasks.workunit.client.0.smithi107.stdout:0/596: dread de/d11/d74/f79 [0,4194304] 0 2024-03-11T19:56:00.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/519: dwrite d10/d19/d3b/f8b [0,4194304] 0 2024-03-11T19:56:00.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/520: chown d10/d16/d24/d2e/f36 3859 1 2024-03-11T19:56:00.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/521: write d10/d19/f61 [9437364,54344] 0 2024-03-11T19:56:00.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/522: chown d10/d16/d24/d2e/d9d/fa8 39 1 2024-03-11T19:56:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:4/586: write d4/d7/d1f/d2b/f5d [1472189,103416] 0 2024-03-11T19:56:00.405 INFO:tasks.workunit.client.0.smithi107.stdout:7/691: dwrite d6/d9/d4e/fa4 [0,4194304] 0 2024-03-11T19:56:00.444 INFO:tasks.workunit.client.0.smithi107.stdout:4/587: dwrite d4/d7/d1f/d2b/d24/d26/fae [0,4194304] 0 2024-03-11T19:56:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/692: dwrite d6/d9/d57/d65/f74 [0,4194304] 0 2024-03-11T19:56:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/693: dread - d6/d21/fcb zero size 2024-03-11T19:56:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/694: write d6/d9/d57/f5a [690438,111563] 0 2024-03-11T19:56:00.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/695: write d6/da/d16/f6f [2027602,63239] 0 2024-03-11T19:56:00.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/696: readlink d6/da/l8e 0 2024-03-11T19:56:00.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/697: stat d6/d9/d57/d65/fc7 0 2024-03-11T19:56:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:4/588: dwrite d4/fc [0,4194304] 0 2024-03-11T19:56:00.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/698: write d6/da/d16/f36 [2442156,9842] 0 2024-03-11T19:56:00.638 INFO:tasks.workunit.client.0.smithi107.stdout:7/699: write d6/d9/d57/f5a [1016002,25442] 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/700: chown d6/da/d16/d5c/c6c 3 1 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/639: rename d1/d10/d13/d6c/d2d/d4d/d91/laa to d1/d10/d2a/d7e/d8b/lc7 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:6/523: creat d10/fad x:0 0 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/701: readlink d6/da/d16/d91/d97/lcd 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/640: write d1/d10/d21/d43/d48/d98/fa8 [805243,16754] 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/641: readlink d1/d10/d13/d6c/d2d/l33 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/642: fdatasync d1/d10/d13/d6c/f44 0 2024-03-11T19:56:00.639 INFO:tasks.workunit.client.0.smithi107.stdout:0/597: truncate de/f18 493140 0 2024-03-11T19:56:00.641 INFO:tasks.workunit.client.0.smithi107.stdout:9/549: sync 2024-03-11T19:56:00.641 INFO:tasks.workunit.client.0.smithi107.stdout:5/685: sync 2024-03-11T19:56:00.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/576: sync 2024-03-11T19:56:00.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/686: truncate da/d14/d18/f2c 2352578 0 2024-03-11T19:56:00.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/687: write da/d14/d18/d1e/d9b/d73/faa [1509574,66409] 0 2024-03-11T19:56:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:7/702: link d6/da/d16/d5c/c6c d6/da/d16/cdf 0 2024-03-11T19:56:00.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/703: dread - d6/d9/d57/d65/fc6 zero size 2024-03-11T19:56:00.645 INFO:tasks.workunit.client.0.smithi107.stdout:2/577: creat d1/d21/d35/d67/fc0 x:0 0 0 2024-03-11T19:56:00.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/688: unlink da/d14/d43/d9e/lb4 0 2024-03-11T19:56:00.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/578: link d1/d1a/d1e/d26/f48 d1/d1a/d34/d4f/d63/db6/fc1 0 2024-03-11T19:56:00.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/579: chown d1/f40 110044957 1 2024-03-11T19:56:00.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/580: write d1/d1a/f65 [910009,44791] 0 2024-03-11T19:56:00.661 INFO:tasks.workunit.client.0.smithi107.stdout:2/581: dread d1/d1a/f20 [0,4194304] 0 2024-03-11T19:56:00.661 INFO:tasks.workunit.client.0.smithi107.stdout:2/582: dread - d1/f7b zero size 2024-03-11T19:56:00.661 INFO:tasks.workunit.client.0.smithi107.stdout:2/583: getdents d1/d1d/d80 0 2024-03-11T19:56:00.672 INFO:tasks.workunit.client.0.smithi107.stdout:6/524: rename d10/d16/f33 to d10/d3d/fae 0 2024-03-11T19:56:00.672 INFO:tasks.workunit.client.0.smithi107.stdout:6/525: readlink d10/d19/l1b 0 2024-03-11T19:56:00.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/526: mknod d10/d16/d24/caf 0 2024-03-11T19:56:00.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/527: chown d10 5199 1 2024-03-11T19:56:00.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/528: chown d10/d16/f84 7991522 1 2024-03-11T19:56:00.676 INFO:tasks.workunit.client.0.smithi107.stdout:9/550: dread d4/d1d/d1f/d28/d2c/d96/fa4 [0,4194304] 0 2024-03-11T19:56:00.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/529: link d10/d16/d1f/d51/f87 d10/d16/d1f/d51/d9c/fb0 0 2024-03-11T19:56:00.679 INFO:tasks.workunit.client.0.smithi107.stdout:9/551: unlink d4/f12 0 2024-03-11T19:56:00.680 INFO:tasks.workunit.client.0.smithi107.stdout:0/598: read de/d7a/f85 [2394074,113007] 0 2024-03-11T19:56:00.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/599: chown de/d11/d23/d32/d34/f6c 4558247 1 2024-03-11T19:56:00.695 INFO:tasks.workunit.client.0.smithi107.stdout:7/704: dread d6/da/d16/d5c/d7d/d60/d61/fb3 [0,4194304] 0 2024-03-11T19:56:00.702 INFO:tasks.workunit.client.0.smithi107.stdout:3/643: dwrite d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:56:00.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/600: unlink de/d11/d23/d32/d34/d39/f5a 0 2024-03-11T19:56:00.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/530: dread d10/d19/d3b/f8b [0,4194304] 0 2024-03-11T19:56:00.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/601: symlink de/d9e/lb4 0 2024-03-11T19:56:00.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/602: truncate de/d11/d23/d26/f43 2756301 0 2024-03-11T19:56:00.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/689: dread da/d14/d18/d1e/d9b/f57 [0,4194304] 0 2024-03-11T19:56:00.707 INFO:tasks.workunit.client.0.smithi107.stdout:0/603: unlink de/d11/d23/d26/c3e 0 2024-03-11T19:56:00.717 INFO:tasks.workunit.client.0.smithi107.stdout:7/705: dread d6/da/f45 [0,4194304] 0 2024-03-11T19:56:00.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/706: truncate d6/d9/f4f 567447 0 2024-03-11T19:56:00.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/531: rename d10/d19/d38/d40 to d10/d19/d38/d60/db1 0 2024-03-11T19:56:00.726 INFO:tasks.workunit.client.0.smithi107.stdout:6/532: creat d10/fb2 x:0 0 0 2024-03-11T19:56:00.727 INFO:tasks.workunit.client.0.smithi107.stdout:2/584: dwrite d1/d21/d35/fa5 [0,4194304] 0 2024-03-11T19:56:00.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/644: write d1/d10/f57 [8213462,5674] 0 2024-03-11T19:56:00.734 INFO:tasks.workunit.client.0.smithi107.stdout:6/533: unlink d10/d16/d1f/d51/f6c 0 2024-03-11T19:56:00.734 INFO:tasks.workunit.client.0.smithi107.stdout:6/534: stat d10/d19/d3b 0 2024-03-11T19:56:00.735 INFO:tasks.workunit.client.0.smithi107.stdout:6/535: truncate d10/d16/d1f/d51/d76/d7f/f97 3718593 0 2024-03-11T19:56:00.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/536: rename d10/d16/d24/d2e/f7e to d10/d16/d2b/d39/fb3 0 2024-03-11T19:56:00.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/537: fsync d10/d16/d1f/f25 0 2024-03-11T19:56:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/585: dread d1/d1d/f3e [0,4194304] 0 2024-03-11T19:56:00.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/586: creat d1/d21/d35/fc2 x:0 0 0 2024-03-11T19:56:00.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/587: creat d1/d1a/d1e/fc3 x:0 0 0 2024-03-11T19:56:00.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/604: write de/d11/f1b [550358,85354] 0 2024-03-11T19:56:00.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/690: write f4 [11535932,65956] 0 2024-03-11T19:56:00.775 INFO:tasks.workunit.client.0.smithi107.stdout:0/605: symlink de/d11/d23/d24/lb5 0 2024-03-11T19:56:00.775 INFO:tasks.workunit.client.0.smithi107.stdout:0/606: chown de/d11/d74/f79 86281 1 2024-03-11T19:56:00.775 INFO:tasks.workunit.client.0.smithi107.stdout:9/552: dwrite d4/d1d/d5c/d64/fa8 [0,4194304] 0 2024-03-11T19:56:00.775 INFO:tasks.workunit.client.0.smithi107.stdout:5/691: rename da/f32 to da/d14/d18/d44/dcd/fd5 0 2024-03-11T19:56:00.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/692: write da/d14/d43/d7f/fc2 [450472,126347] 0 2024-03-11T19:56:00.777 INFO:tasks.workunit.client.0.smithi107.stdout:9/553: creat d4/d9/d9f/da7/fca x:0 0 0 2024-03-11T19:56:00.778 INFO:tasks.workunit.client.0.smithi107.stdout:5/693: creat da/d14/d18/d1e/d9b/d73/fd6 x:0 0 0 2024-03-11T19:56:00.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/554: unlink d4/d1d/lc4 0 2024-03-11T19:56:00.780 INFO:tasks.workunit.client.0.smithi107.stdout:0/607: dread de/d11/d23/d32/d34/f6c [0,4194304] 0 2024-03-11T19:56:00.783 INFO:tasks.workunit.client.0.smithi107.stdout:5/694: truncate da/d14/d18/f51 1603718 0 2024-03-11T19:56:00.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/555: creat d4/fcb x:0 0 0 2024-03-11T19:56:00.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/556: read - d4/d1c/d37/f75 zero size 2024-03-11T19:56:00.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/557: mknod d4/d1d/d1f/ccc 0 2024-03-11T19:56:00.790 INFO:tasks.workunit.client.0.smithi107.stdout:9/558: mkdir d4/d1d/d1f/dcd 0 2024-03-11T19:56:00.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/559: symlink d4/d1d/d1f/d28/d2c/d96/lce 0 2024-03-11T19:56:00.794 INFO:tasks.workunit.client.0.smithi107.stdout:9/560: chown d4/d1d/d1f/d28/c57 755382 1 2024-03-11T19:56:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:8/520: sync 2024-03-11T19:56:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/623: sync 2024-03-11T19:56:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/538: dwrite d10/d16/d24/d2e/f2f [0,4194304] 0 2024-03-11T19:56:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:9/561: unlink d4/d1c/d37/f8f 0 2024-03-11T19:56:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:9/562: fsync d4/d1d/d5c/d64/f84 0 2024-03-11T19:56:00.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/539: mknod d10/d16/d1f/cb4 0 2024-03-11T19:56:00.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/540: write d10/d16/d1f/d51/d9c/fb0 [197569,8119] 0 2024-03-11T19:56:00.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/541: write d10/d16/fab [857120,1066] 0 2024-03-11T19:56:00.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/542: write d10/d16/d1f/f79 [4029119,40425] 0 2024-03-11T19:56:00.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/707: dwrite d6/da/d16/d91/d97/fb2 [0,4194304] 0 2024-03-11T19:56:00.803 INFO:tasks.workunit.client.0.smithi107.stdout:9/563: unlink d4/d1d/d1f/d28/d2c/f46 0 2024-03-11T19:56:00.803 INFO:tasks.workunit.client.0.smithi107.stdout:9/564: write d4/d1d/d1f/f39 [5039305,80365] 0 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/624: dread d6/d7e/d81/d27/f7b [0,4194304] 0 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/625: fsync d6/d7e/d81/d27/d2c/d32/f7f 0 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/626: fsync d6/f54 0 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/627: chown d6/d7e/d81/d27/d2c/d7d/l90 308022298 1 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/628: truncate d6/d86/fbc 1383718 0 2024-03-11T19:56:00.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/565: rename d4/d1d/d1f/d28/c57 to d4/d9/d4d/ccf 0 2024-03-11T19:56:00.816 INFO:tasks.workunit.client.0.smithi107.stdout:9/566: mknod d4/d1d/d1f/d28/d2c/cd0 0 2024-03-11T19:56:00.817 INFO:tasks.workunit.client.0.smithi107.stdout:9/567: write d4/d1d/d4b/f81 [1165578,130708] 0 2024-03-11T19:56:00.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/568: rename d4/d1c/d37/c87 to d4/d1d/d1f/d28/d2c/cd1 0 2024-03-11T19:56:00.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/695: write da/d14/d18/d1e/fb6 [7875806,34866] 0 2024-03-11T19:56:00.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/588: dwrite d1/d38/d75/fb0 [0,4194304] 0 2024-03-11T19:56:00.822 INFO:tasks.workunit.client.0.smithi107.stdout:9/569: fdatasync d4/d1d/d1f/d28/d2c/f62 0 2024-03-11T19:56:00.822 INFO:tasks.workunit.client.0.smithi107.stdout:2/589: rmdir d1/d38 39 2024-03-11T19:56:00.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/629: mkdir d6/d7e/d81/d27/d2c/d32/dc1/dc9 0 2024-03-11T19:56:00.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/630: fsync d6/d7e/d81/d27/d2c/d32/dc1/f9b 0 2024-03-11T19:56:00.825 INFO:tasks.workunit.client.0.smithi107.stdout:7/708: dread d6/f5f [0,4194304] 0 2024-03-11T19:56:00.825 INFO:tasks.workunit.client.0.smithi107.stdout:7/709: fdatasync d6/da/d16/d5c/d7d/d1b/f3c 0 2024-03-11T19:56:00.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/631: creat d6/d7e/d81/d27/d2c/d32/d58/fca x:0 0 0 2024-03-11T19:56:00.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/632: fsync d6/d7e/d81/d27/d2c/d32/dc1/f96 0 2024-03-11T19:56:00.829 INFO:tasks.workunit.client.0.smithi107.stdout:7/710: symlink d6/da/d16/d91/d97/d9b/le0 0 2024-03-11T19:56:00.829 INFO:tasks.workunit.client.0.smithi107.stdout:7/711: stat d6/ldc 0 2024-03-11T19:56:00.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/543: write d10/d19/f31 [3220053,5543] 0 2024-03-11T19:56:00.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/544: fsync d10/d19/d38/f75 0 2024-03-11T19:56:00.839 INFO:tasks.workunit.client.0.smithi107.stdout:6/545: mknod d10/d16/d24/d2e/cb5 0 2024-03-11T19:56:00.848 INFO:tasks.workunit.client.0.smithi107.stdout:0/608: write de/d11/d23/d24/f69 [1176161,46334] 0 2024-03-11T19:56:00.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/645: dwrite d1/d10/d4e/fa2 [0,4194304] 0 2024-03-11T19:56:00.861 INFO:tasks.workunit.client.0.smithi107.stdout:1/633: creat d6/d86/fcb x:0 0 0 2024-03-11T19:56:00.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/546: dwrite fc [4194304,4194304] 0 2024-03-11T19:56:00.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/547: mkdir d10/d16/d1f/d51/d9c/db6 0 2024-03-11T19:56:00.879 INFO:tasks.workunit.client.0.smithi107.stdout:1/634: write d6/d28/f3f [1991259,128464] 0 2024-03-11T19:56:00.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/635: creat d6/d7e/d81/d27/d2c/d32/d58/fcc x:0 0 0 2024-03-11T19:56:00.882 INFO:tasks.workunit.client.0.smithi107.stdout:8/521: dwrite dd/d13/f3d [0,4194304] 0 2024-03-11T19:56:00.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/636: mkdir d6/d7e/d81/d27/d2c/d32/dcd 0 2024-03-11T19:56:00.883 INFO:tasks.workunit.client.0.smithi107.stdout:1/637: truncate d6/d28/f80 593035 0 2024-03-11T19:56:00.883 INFO:tasks.workunit.client.0.smithi107.stdout:8/522: write dd/de/d1b/d4b/f74 [865039,44624] 0 2024-03-11T19:56:00.883 INFO:tasks.workunit.client.0.smithi107.stdout:8/523: chown dd/de/d1b/d20/f6a 74381956 1 2024-03-11T19:56:00.884 INFO:tasks.workunit.client.0.smithi107.stdout:0/609: link de/d11/d74/l96 de/d7a/lb6 0 2024-03-11T19:56:00.888 INFO:tasks.workunit.client.0.smithi107.stdout:2/590: rename d1/d1a/d8e/lb4 to d1/d1a/d34/d4f/lc4 0 2024-03-11T19:56:00.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/638: rename d6/d86/dc0/cc5 to d6/d7e/da4/cce 0 2024-03-11T19:56:00.893 INFO:tasks.workunit.client.0.smithi107.stdout:2/591: rename d1/d1a/d1e/d41/l44 to d1/lc5 0 2024-03-11T19:56:00.893 INFO:tasks.workunit.client.0.smithi107.stdout:2/592: readlink d1/l2 0 2024-03-11T19:56:00.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/593: creat d1/d1a/d34/d4f/fc6 x:0 0 0 2024-03-11T19:56:00.896 INFO:tasks.workunit.client.0.smithi107.stdout:2/594: creat d1/d38/d75/d76/fc7 x:0 0 0 2024-03-11T19:56:00.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/548: dread d10/d19/f74 [4194304,4194304] 0 2024-03-11T19:56:00.908 INFO:tasks.workunit.client.0.smithi107.stdout:6/549: getdents d10/d16/d2b 0 2024-03-11T19:56:00.908 INFO:tasks.workunit.client.0.smithi107.stdout:6/550: write d10/d16/d1f/d51/f87 [1131305,18628] 0 2024-03-11T19:56:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:7/712: dwrite d6/da/d16/f3f [0,4194304] 0 2024-03-11T19:56:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:9/570: dwrite d4/d1d/d5c/d64/fa8 [0,4194304] 0 2024-03-11T19:56:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:9/571: write d4/d9/dae/f94 [1131605,2935] 0 2024-03-11T19:56:00.910 INFO:tasks.workunit.client.0.smithi107.stdout:9/572: rmdir d4/d1c 39 2024-03-11T19:56:00.910 INFO:tasks.workunit.client.0.smithi107.stdout:9/573: write d4/d1c/f9e [186290,44751] 0 2024-03-11T19:56:00.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/551: truncate d10/d19/f23 1389261 0 2024-03-11T19:56:00.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/552: mkdir d10/d16/d1f/d51/db7 0 2024-03-11T19:56:00.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/553: mkdir d10/d19/d38/d60/db8 0 2024-03-11T19:56:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/554: read - d10/d19/d38/f75 zero size 2024-03-11T19:56:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/574: getdents d4/d9/d9f/da7 0 2024-03-11T19:56:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/555: creat d10/d19/d38/fb9 x:0 0 0 2024-03-11T19:56:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/556: write d10/d19/f77 [4976442,46586] 0 2024-03-11T19:56:00.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/557: write d10/d19/d3b/f4e [586321,1411] 0 2024-03-11T19:56:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/558: rmdir d10/d16/d2b/d39/d6a/d6e 39 2024-03-11T19:56:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/559: write d10/d16/d2b/f9a [797179,60369] 0 2024-03-11T19:56:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/560: getdents d10/d19/d38/d60/db8 0 2024-03-11T19:56:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/561: write d10/d19/f5d [60630,119557] 0 2024-03-11T19:56:00.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/610: dwrite de/d11/fb0 [0,4194304] 0 2024-03-11T19:56:00.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/611: creat de/d7a/fb7 x:0 0 0 2024-03-11T19:56:00.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/612: creat de/d11/d23/d24/da4/dae/fb8 x:0 0 0 2024-03-11T19:56:00.938 INFO:tasks.workunit.client.0.smithi107.stdout:3/646: dwrite d1/d45/fbf [0,4194304] 0 2024-03-11T19:56:00.938 INFO:tasks.workunit.client.0.smithi107.stdout:3/647: fdatasync d1/d10/d4e/d73/fc3 0 2024-03-11T19:56:00.944 INFO:tasks.workunit.client.0.smithi107.stdout:3/648: creat d1/d10/d13/d6c/d5d/d76/fc8 x:0 0 0 2024-03-11T19:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/649: getdents d1 0 2024-03-11T19:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/650: write d1/d10/d13/d4a/d4c/f2f [30380,37601] 0 2024-03-11T19:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/651: fdatasync d1/d10/d21/d43/d48/f50 0 2024-03-11T19:56:00.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/639: dwrite d6/d28/f3f [0,4194304] 0 2024-03-11T19:56:00.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/640: fdatasync d6/d7e/d81/d27/f7b 0 2024-03-11T19:56:00.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/641: symlink d6/d7e/d81/d27/lcf 0 2024-03-11T19:56:00.969 INFO:tasks.workunit.client.0.smithi107.stdout:9/575: dread d4/d9/dae/f77 [0,4194304] 0 2024-03-11T19:56:00.969 INFO:tasks.workunit.client.0.smithi107.stdout:1/642: creat d6/d7e/d81/d27/d2c/d32/d58/fd0 x:0 0 0 2024-03-11T19:56:00.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/713: dwrite d6/da/d16/d5c/d7d/fa9 [0,4194304] 0 2024-03-11T19:56:00.970 INFO:tasks.workunit.client.0.smithi107.stdout:1/643: mknod d6/d7e/d81/d27/d2c/d32/dcd/cd1 0 2024-03-11T19:56:00.970 INFO:tasks.workunit.client.0.smithi107.stdout:0/613: rename de/d11/d23/d32/d34/d39/l52 to de/d7a/lb9 0 2024-03-11T19:56:00.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/644: creat d6/d7e/d81/d27/d2c/d72/fd2 x:0 0 0 2024-03-11T19:56:00.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/645: fsync d6/d5b/f78 0 2024-03-11T19:56:00.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/646: write d6/d7e/d81/d27/d2c/d32/d58/fca [30105,109174] 0 2024-03-11T19:56:00.973 INFO:tasks.workunit.client.0.smithi107.stdout:1/647: mkdir d6/d7e/da5/dac/dd3 0 2024-03-11T19:56:00.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/652: write d1/d10/d26/f25 [4503814,89562] 0 2024-03-11T19:56:00.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/614: mkdir de/d11/d74/dba 0 2024-03-11T19:56:00.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/615: link de/d11/d23/d32/d34/d39/d55/l5c de/d9e/lbb 0 2024-03-11T19:56:01.005 INFO:tasks.workunit.client.0.smithi107.stdout:7/714: dread d6/d9/d4e/fa4 [0,4194304] 0 2024-03-11T19:56:01.006 INFO:tasks.workunit.client.0.smithi107.stdout:7/715: symlink d6/da/d4d/le1 0 2024-03-11T19:56:01.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:00 smithi122 ceph-mon[36397]: pgmap v80: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 145 MiB/s wr, 630 op/s 2024-03-11T19:56:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/576: dwrite d4/d1d/d5c/d97/fb9 [4194304,4194304] 0 2024-03-11T19:56:01.026 INFO:tasks.workunit.client.0.smithi107.stdout:1/648: dwrite d6/f1d [4194304,4194304] 0 2024-03-11T19:56:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:1/649: write d6/d7e/d81/f4e [790052,68657] 0 2024-03-11T19:56:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:1/650: truncate d6/d7e/d81/d27/d2c/d32/d58/fca 501996 0 2024-03-11T19:56:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:7/716: read d6/da/fbb [3191865,9813] 0 2024-03-11T19:56:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:7/717: truncate d6/d9/d57/d65/fc7 483526 0 2024-03-11T19:56:01.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/651: rename d6/d7e/da5/dac to d6/d7e/d81/d27/d2c/d72/dc3/dd4 0 2024-03-11T19:56:01.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/652: truncate d6/d7e/d81/d27/d2c/d32/dc1/f63 3505147 0 2024-03-11T19:56:01.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/653: write d6/d19/f73 [4214945,89047] 0 2024-03-11T19:56:01.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:00 smithi145 ceph-mon[36730]: pgmap v80: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 145 MiB/s wr, 630 op/s 2024-03-11T19:56:01.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/718: truncate d6/da/d16/d5c/d7d/d60/d61/fb3 339508 0 2024-03-11T19:56:01.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/653: dwrite d1/d10/d2a/d7e/d8b/f97 [0,4194304] 0 2024-03-11T19:56:01.038 INFO:tasks.workunit.client.0.smithi107.stdout:1/654: truncate d6/d28/f41 2956062 0 2024-03-11T19:56:01.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:00 smithi107 ceph-mon[31712]: pgmap v80: 129 pgs: 129 active+clean; 960 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 122 MiB/s rd, 145 MiB/s wr, 630 op/s 2024-03-11T19:56:01.043 INFO:tasks.workunit.client.0.smithi107.stdout:3/654: mknod d1/d10/d13/d6c/d2d/d4d/d91/da5/cc9 0 2024-03-11T19:56:01.044 INFO:tasks.workunit.client.0.smithi107.stdout:3/655: mkdir d1/d10/d13/d4a/d4c/d49/dca 0 2024-03-11T19:56:01.046 INFO:tasks.workunit.client.0.smithi107.stdout:3/656: unlink d1/d10/l1c 0 2024-03-11T19:56:01.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/657: rename d1/d10/d13/l34 to d1/d10/lcb 0 2024-03-11T19:56:01.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/658: write d1/d10/d26/f58 [780057,4915] 0 2024-03-11T19:56:01.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/719: dread d6/da/d4d/f71 [0,4194304] 0 2024-03-11T19:56:01.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/720: rename d6/da/d16/d91/dc8/c88 to d6/da/d16/d91/d97/ce2 0 2024-03-11T19:56:01.062 INFO:tasks.workunit.client.0.smithi107.stdout:7/721: read - d6/d9/d57/d65/f99 zero size 2024-03-11T19:56:01.062 INFO:tasks.workunit.client.0.smithi107.stdout:7/722: write d6/da/fbc [234754,71608] 0 2024-03-11T19:56:01.062 INFO:tasks.workunit.client.0.smithi107.stdout:7/723: write d6/da/d49/f4b [910871,50741] 0 2024-03-11T19:56:01.062 INFO:tasks.workunit.client.0.smithi107.stdout:7/724: write d6/d9/fa6 [1036904,42967] 0 2024-03-11T19:56:01.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/725: unlink d6/da/d16/d5c/d7d/d1b/f3c 0 2024-03-11T19:56:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/589: sync 2024-03-11T19:56:01.127 INFO:tasks.workunit.client.0.smithi107.stdout:7/726: dwrite d6/da/d16/d5c/d7d/d60/f93 [0,4194304] 0 2024-03-11T19:56:01.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/577: dwrite d4/d9/dae/f7d [0,4194304] 0 2024-03-11T19:56:01.128 INFO:tasks.workunit.client.0.smithi107.stdout:1/655: dread d6/d7e/d81/d27/f7b [0,4194304] 0 2024-03-11T19:56:01.128 INFO:tasks.workunit.client.0.smithi107.stdout:3/659: dwrite d1/d10/d21/d43/f84 [0,4194304] 0 2024-03-11T19:56:01.129 INFO:tasks.workunit.client.0.smithi107.stdout:1/656: creat d6/d5b/fd5 x:0 0 0 2024-03-11T19:56:01.134 INFO:tasks.workunit.client.0.smithi107.stdout:9/578: readlink d4/d9/dae/d6d/l85 0 2024-03-11T19:56:01.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/657: chown d6/d28/dc4/la1 53 1 2024-03-11T19:56:01.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/658: dread - d6/d7e/d81/d27/d2c/f6a zero size 2024-03-11T19:56:01.138 INFO:tasks.workunit.client.0.smithi107.stdout:3/660: mkdir d1/d10/d13/d6c/d2d/d4d/d91/dcc 0 2024-03-11T19:56:01.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/659: mkdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/dd6 0 2024-03-11T19:56:01.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/660: readlink d6/l23 0 2024-03-11T19:56:01.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/661: dread - d6/d7e/d81/d27/d2c/fb1 zero size 2024-03-11T19:56:01.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/590: dwrite d4/d7/f8 [0,4194304] 0 2024-03-11T19:56:01.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/662: rmdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3 39 2024-03-11T19:56:01.142 INFO:tasks.workunit.client.0.smithi107.stdout:4/591: link d4/d7/d1f/d2b/d19/fa7 d4/d7/da0/fb9 0 2024-03-11T19:56:01.142 INFO:tasks.workunit.client.0.smithi107.stdout:1/663: creat d6/d7e/d81/d27/d2c/d7d/fd7 x:0 0 0 2024-03-11T19:56:01.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/664: write d6/d7e/d81/f29 [1859632,106774] 0 2024-03-11T19:56:01.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/665: write d6/d86/fcb [984529,92577] 0 2024-03-11T19:56:01.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/592: creat d4/d7/da0/fba x:0 0 0 2024-03-11T19:56:01.144 INFO:tasks.workunit.client.0.smithi107.stdout:1/666: mkdir d6/d7e/d81/d27/d2c/d7d/dd8 0 2024-03-11T19:56:01.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/593: mkdir d4/d7/d2f/d33/d3a/d79/dbb 0 2024-03-11T19:56:01.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/594: chown d4/d7/d1f/d2b/d93 195763 1 2024-03-11T19:56:01.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/595: rmdir d4/d7/db/d5e 39 2024-03-11T19:56:01.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/579: write d4/d1d/d1f/d28/d2c/d96/fa4 [4153166,86827] 0 2024-03-11T19:56:01.161 INFO:tasks.workunit.client.0.smithi107.stdout:9/580: mknod d4/d1d/d5c/d97/cd2 0 2024-03-11T19:56:01.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/581: mkdir d4/d1d/d1f/d28/d2c/d96/dd3 0 2024-03-11T19:56:01.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/582: chown d4/d1c/l71 904187 1 2024-03-11T19:56:01.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/583: getdents d4/d1d/d1f/d78 0 2024-03-11T19:56:01.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/584: chown d4/d1d/d1f/d28/fc1 6236 1 2024-03-11T19:56:01.178 INFO:tasks.workunit.client.0.smithi107.stdout:9/585: dread d4/d1d/d1f/d28/fa2 [0,4194304] 0 2024-03-11T19:56:01.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/661: dwrite d1/d10/d13/d6c/f44 [0,4194304] 0 2024-03-11T19:56:01.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/662: chown d1/d10/d13/da1 73255867 1 2024-03-11T19:56:01.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/586: getdents d4/d9/dae/d6d 0 2024-03-11T19:56:01.183 INFO:tasks.workunit.client.0.smithi107.stdout:7/727: dwrite d6/d9/d57/fda [0,4194304] 0 2024-03-11T19:56:01.185 INFO:tasks.workunit.client.0.smithi107.stdout:7/728: link d6/d9/dae/cb5 d6/d9/d57/ce3 0 2024-03-11T19:56:01.185 INFO:tasks.workunit.client.0.smithi107.stdout:7/729: write d6/d9/d57/da2/fcc [4651677,84792] 0 2024-03-11T19:56:01.187 INFO:tasks.workunit.client.0.smithi107.stdout:7/730: creat d6/da/d16/d91/d97/d9b/fe4 x:0 0 0 2024-03-11T19:56:01.187 INFO:tasks.workunit.client.0.smithi107.stdout:7/731: truncate d6/f7e 5809066 0 2024-03-11T19:56:01.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/732: mknod d6/da/d16/d5c/d7d/d60/d61/db9/ce5 0 2024-03-11T19:56:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/733: dread d6/d9/d57/d65/fc7 [0,4194304] 0 2024-03-11T19:56:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/734: mkdir d6/da/d16/d5c/d7d/d1b/de6 0 2024-03-11T19:56:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/667: dwrite d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82/f9a [0,4194304] 0 2024-03-11T19:56:01.206 INFO:tasks.workunit.client.0.smithi107.stdout:7/735: read d6/d9/f4f [468780,7401] 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/596: dwrite d4/d7/da0/fa1 [0,4194304] 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/597: write d4/d7/d1f/d2b/d19/d75/d8e/fa8 [3174108,98524] 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/598: fsync f3 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/663: dread d1/d10/d4e/d73/d8d/f95 [0,4194304] 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/736: mknod d6/da/d4d/ce7 0 2024-03-11T19:56:01.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/737: fsync d6/da/d16/d5c/d7d/d60/fc3 0 2024-03-11T19:56:01.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/599: creat d4/d7/d1f/d2b/d19/d75/d8e/fbc x:0 0 0 2024-03-11T19:56:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/696: sync 2024-03-11T19:56:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/697: write da/d14/d43/d47/fc7 [247229,36882] 0 2024-03-11T19:56:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/698: write da/fb [2827294,79713] 0 2024-03-11T19:56:01.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/699: symlink da/d14/d18/d1e/d9b/d72/dc8/ld7 0 2024-03-11T19:56:01.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/700: rename da/d14/d43/lbc to da/d14/d38/d59/dbd/ld8 0 2024-03-11T19:56:01.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/668: dread d6/d28/f4b [4194304,4194304] 0 2024-03-11T19:56:01.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/669: readlink d6/d7e/d81/d27/lbe 0 2024-03-11T19:56:01.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/600: dread f0 [0,4194304] 0 2024-03-11T19:56:01.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/601: mknod d4/d7/da0/cbd 0 2024-03-11T19:56:01.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/602: fsync d4/d7/da0/fb9 0 2024-03-11T19:56:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/587: dwrite d4/d1d/d1f/d78/fc0 [0,4194304] 0 2024-03-11T19:56:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/588: write d4/f53 [3300010,6786] 0 2024-03-11T19:56:01.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/589: dread - d4/d1d/d5c/d97/faa zero size 2024-03-11T19:56:01.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/664: dwrite d1/f9d [0,4194304] 0 2024-03-11T19:56:01.259 INFO:tasks.workunit.client.0.smithi107.stdout:9/590: dread d4/d9/dae/f77 [0,4194304] 0 2024-03-11T19:56:01.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/591: creat d4/d9/d4d/fd4 x:0 0 0 2024-03-11T19:56:01.263 INFO:tasks.workunit.client.0.smithi107.stdout:9/592: symlink d4/d9/d9f/ld5 0 2024-03-11T19:56:01.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/670: dwrite d6/f1b [0,4194304] 0 2024-03-11T19:56:01.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/593: unlink d4/d9/c3a 0 2024-03-11T19:56:01.270 INFO:tasks.workunit.client.0.smithi107.stdout:5/701: dwrite da/d31/d42/f1f [0,4194304] 0 2024-03-11T19:56:01.270 INFO:tasks.workunit.client.0.smithi107.stdout:5/702: chown da/d14/d38/d59/dbd/ld8 415 1 2024-03-11T19:56:01.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/594: write d4/d1d/d1f/d28/f67 [685459,47516] 0 2024-03-11T19:56:01.273 INFO:tasks.workunit.client.0.smithi107.stdout:9/595: rename d4/d1d/d4b/f81 to d4/d9/d4d/fd6 0 2024-03-11T19:56:01.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/596: rmdir d4/d1d/d1f/dcd 0 2024-03-11T19:56:01.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/597: rename d4/d1c/d47/l9b to d4/d1c/d37/ld7 0 2024-03-11T19:56:01.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/598: mkdir d4/d9/d9f/dd8 0 2024-03-11T19:56:01.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/599: write d4/d1d/d1f/d78/f1a [1734935,26409] 0 2024-03-11T19:56:01.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/600: mkdir d4/d1c/d37/dd9 0 2024-03-11T19:56:01.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/601: link d4/d1d/d1f/d28/f42 d4/d9/d9f/da7/fda 0 2024-03-11T19:56:01.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/602: getdents d4/d1d/d4b 0 2024-03-11T19:56:01.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/603: dread - d4/d1d/d5c/fc7 zero size 2024-03-11T19:56:01.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/604: fdatasync d4/d9/dae/d6d/f2d 0 2024-03-11T19:56:01.288 INFO:tasks.workunit.client.0.smithi107.stdout:3/665: dwrite d1/d10/d13/f61 [0,4194304] 0 2024-03-11T19:56:01.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/605: rmdir d4 39 2024-03-11T19:56:01.323 INFO:tasks.workunit.client.0.smithi107.stdout:9/606: dwrite d4/d9/dae/f7d [0,4194304] 0 2024-03-11T19:56:01.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/607: truncate d4/d9/dae/f7d 303695 0 2024-03-11T19:56:01.329 INFO:tasks.workunit.client.0.smithi107.stdout:9/608: truncate d4/d9/d4d/f8e 2296689 0 2024-03-11T19:56:01.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/609: readlink d4/d9/d9f/ld5 0 2024-03-11T19:56:01.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/666: dwrite d1/d10/d4e/fa2 [0,4194304] 0 2024-03-11T19:56:01.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/667: fsync d1/d10/d4e/d73/d8d/f95 0 2024-03-11T19:56:01.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/668: fsync d1/d10/d13/d4a/f93 0 2024-03-11T19:56:01.342 INFO:tasks.workunit.client.0.smithi107.stdout:3/669: unlink d1/d10/d4e/d73/f41 0 2024-03-11T19:56:01.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/670: dread d1/d10/f57 [8388608,4194304] 0 2024-03-11T19:56:01.377 INFO:tasks.workunit.client.0.smithi107.stdout:8/524: sync 2024-03-11T19:56:01.379 INFO:tasks.workunit.client.0.smithi107.stdout:8/525: symlink dd/de/d49/d46/l93 0 2024-03-11T19:56:01.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/610: dwrite d4/d1d/d5c/fc7 [0,4194304] 0 2024-03-11T19:56:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:8/526: rename dd/d13/d32/f8b to dd/d13/d18/d81/f94 0 2024-03-11T19:56:01.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/527: mknod dd/de/d1b/d2c/c95 0 2024-03-11T19:56:01.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/528: chown dd/d13/d18/f5c 27210 1 2024-03-11T19:56:01.397 INFO:tasks.workunit.client.0.smithi107.stdout:3/671: dwrite d1/d10/d13/d4a/d4c/f7b [0,4194304] 0 2024-03-11T19:56:01.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/672: mkdir d1/d10/d13/d6c/d2d/d4d/d91/dcd 0 2024-03-11T19:56:01.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/673: fsync d1/d45/fbf 0 2024-03-11T19:56:01.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/674: fsync d1/d10/d13/d6c/f32 0 2024-03-11T19:56:01.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/675: rename d1/d10/d21/d43/d48/f9f to d1/d10/d13/d4a/d4c/d49/dca/fce 0 2024-03-11T19:56:01.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/676: symlink d1/d10/d13/d6c/d2d/d4d/lcf 0 2024-03-11T19:56:01.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/677: rename d1/d10/d21 to d1/d10/d21/d43/d82/dd0 22 2024-03-11T19:56:01.407 INFO:tasks.workunit.client.0.smithi107.stdout:3/678: chown d1/f6e 9476 1 2024-03-11T19:56:01.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/611: dwrite d4/d9/d9f/fbc [0,4194304] 0 2024-03-11T19:56:01.446 INFO:tasks.workunit.client.0.smithi107.stdout:3/679: dwrite d1/d10/d13/f61 [4194304,4194304] 0 2024-03-11T19:56:01.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/680: creat d1/d10/d4e/d73/d8d/fd1 x:0 0 0 2024-03-11T19:56:01.450 INFO:tasks.workunit.client.0.smithi107.stdout:3/681: rename d1/d10/d26/d17/c46 to d1/d10/d13/d6c/d2d/d4d/d91/da5/cd2 0 2024-03-11T19:56:01.450 INFO:tasks.workunit.client.0.smithi107.stdout:3/682: fsync d1/f6e 0 2024-03-11T19:56:01.450 INFO:tasks.workunit.client.0.smithi107.stdout:3/683: write d1/d10/d13/d6c/f32 [2981891,112958] 0 2024-03-11T19:56:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:2/595: sync 2024-03-11T19:56:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:6/562: sync 2024-03-11T19:56:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/616: sync 2024-03-11T19:56:01.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/596: creat d1/d38/fc8 x:0 0 0 2024-03-11T19:56:01.455 INFO:tasks.workunit.client.0.smithi107.stdout:2/597: creat d1/d21/d35/fc9 x:0 0 0 2024-03-11T19:56:01.455 INFO:tasks.workunit.client.0.smithi107.stdout:2/598: chown d1/d1a/d34/d4f 10 1 2024-03-11T19:56:01.455 INFO:tasks.workunit.client.0.smithi107.stdout:2/599: fdatasync d1/f36 0 2024-03-11T19:56:01.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/617: creat de/d11/d74/fbc x:0 0 0 2024-03-11T19:56:01.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/618: dread de/d11/d23/d32/d34/d39/d3a/d72/f8a [0,4194304] 0 2024-03-11T19:56:01.471 INFO:tasks.workunit.client.0.smithi107.stdout:0/619: rename de/d11/d23/d24/f65 to de/d11/d23/d24/da4/fbd 0 2024-03-11T19:56:01.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/612: dread d4/d1d/d1f/d28/d2c/d96/fa4 [0,4194304] 0 2024-03-11T19:56:01.496 INFO:tasks.workunit.client.0.smithi107.stdout:6/563: dread d10/d16/d1f/f79 [0,4194304] 0 2024-03-11T19:56:01.499 INFO:tasks.workunit.client.0.smithi107.stdout:6/564: mkdir d10/d16/d1f/d51/d7a/dba 0 2024-03-11T19:56:01.504 INFO:tasks.workunit.client.0.smithi107.stdout:9/613: truncate d4/d1d/d5c/d97/fbd 4989753 0 2024-03-11T19:56:01.505 INFO:tasks.workunit.client.0.smithi107.stdout:3/684: dwrite d1/d10/d2a/d7e/d8b/fc6 [0,4194304] 0 2024-03-11T19:56:01.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/620: dwrite de/d11/d23/d32/f54 [0,4194304] 0 2024-03-11T19:56:01.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/600: dwrite d1/d21/d35/d67/f8c [0,4194304] 0 2024-03-11T19:56:01.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/703: sync 2024-03-11T19:56:01.530 INFO:tasks.workunit.client.0.smithi107.stdout:4/603: sync 2024-03-11T19:56:01.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/704: chown da/d14/d38/d59/d5f/f6a 117182205 1 2024-03-11T19:56:01.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/705: write da/d14/d18/d1e/d9b/d72/fac [748205,42174] 0 2024-03-11T19:56:01.534 INFO:tasks.workunit.client.0.smithi107.stdout:8/529: sync 2024-03-11T19:56:01.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/706: mkdir da/d14/d38/d46/dd9 0 2024-03-11T19:56:01.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/707: readlink da/d14/d38/l62 0 2024-03-11T19:56:01.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/671: sync 2024-03-11T19:56:01.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/738: sync 2024-03-11T19:56:01.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/565: dread d10/d19/f77 [0,4194304] 0 2024-03-11T19:56:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/739: rename d6/da/d16/d5c/d7d/d1b/de6 to d6/da/d16/d5c/d7d/d60/dd2/de8 0 2024-03-11T19:56:01.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/566: mkdir d10/d19/d3b/d70/dbb 0 2024-03-11T19:56:01.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/567: stat d10/d19/d3b/d70/d8a 0 2024-03-11T19:56:01.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/614: dwrite d4/d9/f86 [0,4194304] 0 2024-03-11T19:56:01.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/615: stat d4/d1d/d1f 0 2024-03-11T19:56:01.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/616: fdatasync d4/d9/d9f/da7/fca 0 2024-03-11T19:56:01.557 INFO:tasks.workunit.client.0.smithi107.stdout:7/740: link d6/d9/d4e/l50 d6/d9/le9 0 2024-03-11T19:56:01.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/685: dwrite d1/d10/d4e/d73/d8d/f95 [0,4194304] 0 2024-03-11T19:56:01.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/617: unlink d4/d1d/d1f/d78/fc0 0 2024-03-11T19:56:01.563 INFO:tasks.workunit.client.0.smithi107.stdout:7/741: unlink d6/da/d16/d5c/f54 0 2024-03-11T19:56:01.565 INFO:tasks.workunit.client.0.smithi107.stdout:3/686: creat d1/d10/d4e/fd3 x:0 0 0 2024-03-11T19:56:01.568 INFO:tasks.workunit.client.0.smithi107.stdout:3/687: creat d1/d10/d13/d6c/fd4 x:0 0 0 2024-03-11T19:56:01.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/568: rename cf to d10/d16/d24/cbc 0 2024-03-11T19:56:01.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/742: read d6/da/d16/d91/dc8/d80/fb0 [2354228,127443] 0 2024-03-11T19:56:01.597 INFO:tasks.workunit.client.0.smithi107.stdout:4/604: write d4/d7/d1f/d2b/d24/d26/d29/d36/f78 [2933131,87882] 0 2024-03-11T19:56:01.597 INFO:tasks.workunit.client.0.smithi107.stdout:4/605: fdatasync d4/d7/d1f/d2b/d19/f2a 0 2024-03-11T19:56:01.598 INFO:tasks.workunit.client.0.smithi107.stdout:4/606: stat d4/d7/d1f/d2b/d24/d26/d29/l48 0 2024-03-11T19:56:01.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/601: dwrite d1/d1a/d34/d4f/d63/f66 [0,4194304] 0 2024-03-11T19:56:01.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/602: write d1/d38/d75/f84 [4278575,7243] 0 2024-03-11T19:56:01.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/603: stat d1/d1a/d1e/d55/f87 0 2024-03-11T19:56:01.599 INFO:tasks.workunit.client.0.smithi107.stdout:3/688: rename d1/d10/d4e/d73/f20 to d1/d10/d13/d6c/d2d/d4d/d91/da5/fd5 0 2024-03-11T19:56:01.600 INFO:tasks.workunit.client.0.smithi107.stdout:6/569: mkdir d10/d16/d1f/d51/d76/d7f/dbd 0 2024-03-11T19:56:01.602 INFO:tasks.workunit.client.0.smithi107.stdout:6/570: symlink d10/d19/d3b/d70/lbe 0 2024-03-11T19:56:01.602 INFO:tasks.workunit.client.0.smithi107.stdout:6/571: stat d10/d19/f77 0 2024-03-11T19:56:01.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/621: dwrite de/d11/d23/d24/f2d [0,4194304] 0 2024-03-11T19:56:01.605 INFO:tasks.workunit.client.0.smithi107.stdout:0/622: chown de/f88 10924575 1 2024-03-11T19:56:01.606 INFO:tasks.workunit.client.0.smithi107.stdout:4/607: creat d4/fbe x:0 0 0 2024-03-11T19:56:01.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/672: dread d6/d7e/d81/d27/d2c/d32/dc1/f89 [0,4194304] 0 2024-03-11T19:56:01.609 INFO:tasks.workunit.client.0.smithi107.stdout:8/530: dwrite dd/de/d49/d46/f4f [0,4194304] 0 2024-03-11T19:56:01.610 INFO:tasks.workunit.client.0.smithi107.stdout:2/604: rename d1/d1a/d34/d4f/fc6 to d1/d1a/d1e/d55/fca 0 2024-03-11T19:56:01.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/531: mkdir dd/de/d1b/d2c/d90/d96 0 2024-03-11T19:56:01.611 INFO:tasks.workunit.client.0.smithi107.stdout:2/605: symlink d1/d38/lcb 0 2024-03-11T19:56:01.611 INFO:tasks.workunit.client.0.smithi107.stdout:2/606: write d1/d1a/d1e/d26/f30 [795410,18478] 0 2024-03-11T19:56:01.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/607: unlink d1/d21/d35/fc9 0 2024-03-11T19:56:01.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/532: creat dd/de/d24/f97 x:0 0 0 2024-03-11T19:56:01.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/708: dwrite da/f71 [0,4194304] 0 2024-03-11T19:56:01.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/608: dread d4/d7/d1f/d2b/d24/d26/fae [0,4194304] 0 2024-03-11T19:56:01.649 INFO:tasks.workunit.client.0.smithi107.stdout:5/709: truncate da/d31/d42/f13 2009565 0 2024-03-11T19:56:01.649 INFO:tasks.workunit.client.0.smithi107.stdout:5/710: readlink da/d14/d43/d9e/dad/lbe 0 2024-03-11T19:56:01.649 INFO:tasks.workunit.client.0.smithi107.stdout:5/711: write da/d14/d18/d1e/d9b/d73/f8a [634936,55539] 0 2024-03-11T19:56:01.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/609: getdents d4/d7/d1f/d2b/d24 0 2024-03-11T19:56:01.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/610: creat d4/d7/d2f/d33/d3a/fbf x:0 0 0 2024-03-11T19:56:01.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/533: read dd/de/d49/d46/f4f [520849,21039] 0 2024-03-11T19:56:01.650 INFO:tasks.workunit.client.0.smithi107.stdout:4/611: rename d4/l95 to d4/d7/d2f/d33/lc0 0 2024-03-11T19:56:01.650 INFO:tasks.workunit.client.0.smithi107.stdout:4/612: chown d4/d7/d2f/c76 0 1 2024-03-11T19:56:01.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/572: dwrite d10/d19/f23 [0,4194304] 0 2024-03-11T19:56:01.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/573: getdents d10/d16/d2b/da5 0 2024-03-11T19:56:01.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/574: chown d10/d16/f1e 2124561291 1 2024-03-11T19:56:01.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/613: link d4/d7/da0/fb4 d4/d7/d1f/d2b/d19/d9a/fc1 0 2024-03-11T19:56:01.657 INFO:tasks.workunit.client.0.smithi107.stdout:9/618: dwrite d4/d9/dae/f7d [0,4194304] 0 2024-03-11T19:56:01.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/614: creat d4/d7/db/fc2 x:0 0 0 2024-03-11T19:56:01.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/615: fsync d4/d7/d1f/d82/f83 0 2024-03-11T19:56:01.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/534: dread dd/f2e [0,4194304] 0 2024-03-11T19:56:01.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/535: fdatasync dd/de/d1b/f1d 0 2024-03-11T19:56:01.660 INFO:tasks.workunit.client.0.smithi107.stdout:9/619: mkdir d4/d9/d9f/da7/ddb 0 2024-03-11T19:56:01.662 INFO:tasks.workunit.client.0.smithi107.stdout:4/616: mkdir d4/d7/d1f/d2b/d93/dc3 0 2024-03-11T19:56:01.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/712: dread da/f86 [0,4194304] 0 2024-03-11T19:56:01.663 INFO:tasks.workunit.client.0.smithi107.stdout:1/673: dwrite d6/d86/fbc [0,4194304] 0 2024-03-11T19:56:01.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/536: getdents dd/d16 0 2024-03-11T19:56:01.667 INFO:tasks.workunit.client.0.smithi107.stdout:1/674: mkdir d6/d7e/da5/dd9 0 2024-03-11T19:56:01.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/617: creat d4/d7/db/d5c/fc4 x:0 0 0 2024-03-11T19:56:01.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/618: truncate d4/d7/d1f/d2b/d19/d75/d8e/da5/fa6 40961 0 2024-03-11T19:56:01.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/713: stat da/d14/d38/d59/d5f/l7e 0 2024-03-11T19:56:01.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/714: read - da/d14/d18/d1e/d9b/d73/fd6 zero size 2024-03-11T19:56:01.680 INFO:tasks.workunit.client.0.smithi107.stdout:9/620: mkdir d4/d1d/d5c/d97/ddc 0 2024-03-11T19:56:01.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/621: getdents d4/d1c/d37/dd9 0 2024-03-11T19:56:01.687 INFO:tasks.workunit.client.0.smithi107.stdout:3/689: dwrite d1/d10/d21/d43/d48/f56 [0,4194304] 0 2024-03-11T19:56:01.687 INFO:tasks.workunit.client.0.smithi107.stdout:3/690: fsync d1/f6e 0 2024-03-11T19:56:01.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/608: dread d1/d1a/d34/d4f/f83 [0,4194304] 0 2024-03-11T19:56:01.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/623: dwrite de/d11/d23/f9d [0,4194304] 0 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/624: chown l1 160510197 1 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/625: chown de/d11/d23/d32/d34/d39 58 1 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:3/691: dread d1/d10/f57 [4194304,4194304] 0 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/537: read dd/d16/f33 [3800599,101937] 0 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/538: readlink dd/de/d1b/d2c/l30 0 2024-03-11T19:56:01.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/539: creat dd/d13/d32/f98 x:0 0 0 2024-03-11T19:56:01.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/540: chown dd/l12 449 1 2024-03-11T19:56:01.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/541: chown c8 52228 1 2024-03-11T19:56:01.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/542: write dd/de/f48 [1253993,30174] 0 2024-03-11T19:56:01.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/543: chown dd/de/d1b/c77 19491 1 2024-03-11T19:56:01.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/544: truncate dd/d16/f33 234194 0 2024-03-11T19:56:01.722 INFO:tasks.workunit.client.0.smithi107.stdout:8/545: creat dd/f99 x:0 0 0 2024-03-11T19:56:01.727 INFO:tasks.workunit.client.0.smithi107.stdout:6/575: dwrite d10/d16/d2b/d39/fb3 [0,4194304] 0 2024-03-11T19:56:01.727 INFO:tasks.workunit.client.0.smithi107.stdout:6/576: truncate d10/d19/d3b/f99 4400026 0 2024-03-11T19:56:01.729 INFO:tasks.workunit.client.0.smithi107.stdout:6/577: rmdir d10/d16/d2b/d39/d6a 39 2024-03-11T19:56:01.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/622: write d4/d1d/d5c/d64/fa8 [3596618,67120] 0 2024-03-11T19:56:01.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/623: mknod d4/d9/d9f/cdd 0 2024-03-11T19:56:01.751 INFO:tasks.workunit.client.0.smithi107.stdout:4/619: dwrite d4/fbe [0,4194304] 0 2024-03-11T19:56:01.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/715: dread da/d31/f75 [0,4194304] 0 2024-03-11T19:56:01.752 INFO:tasks.workunit.client.0.smithi107.stdout:3/692: write d1/d10/d4e/d73/d8d/f95 [1563726,67235] 0 2024-03-11T19:56:01.753 INFO:tasks.workunit.client.0.smithi107.stdout:1/675: dwrite d6/d7e/d81/d27/d2c/d32/d58/fca [0,4194304] 0 2024-03-11T19:56:01.753 INFO:tasks.workunit.client.0.smithi107.stdout:1/676: readlink d6/d7e/d81/d27/d2c/d32/l6e 0 2024-03-11T19:56:01.755 INFO:tasks.workunit.client.0.smithi107.stdout:8/546: dread dd/d13/d18/f37 [0,4194304] 0 2024-03-11T19:56:01.758 INFO:tasks.workunit.client.0.smithi107.stdout:6/578: rename d10/d16/d24/caf to d10/d16/d1f/d51/d9c/db6/cbf 0 2024-03-11T19:56:01.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/579: mknod d10/d19/d38/d60/db8/cc0 0 2024-03-11T19:56:01.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/580: dread - d10/d3d/d54/fa2 zero size 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/624: dread d4/d1d/d1f/d28/d2c/f49 [0,4194304] 0 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/625: readlink d4/d1d/d1f/d28/d2c/d96/la0 0 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/626: dread - d4/d1d/d1f/d28/f91 zero size 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/620: mkdir d4/d7/d1f/d2b/d24/d26/d29/dc5 0 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/621: chown d4/d7/d2f/d33/d3a/l55 2409 1 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/622: read - d4/d7/d1f/d2b/d24/d26/d29/fa4 zero size 2024-03-11T19:56:01.761 INFO:tasks.workunit.client.0.smithi107.stdout:6/581: unlink d10/d19/l1b 0 2024-03-11T19:56:01.762 INFO:tasks.workunit.client.0.smithi107.stdout:5/716: symlink da/d14/d38/dd0/lda 0 2024-03-11T19:56:01.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/623: truncate d4/d7/d2f/d3c/f91 292798 0 2024-03-11T19:56:01.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/624: dread - d4/d7/db/d5c/f8a zero size 2024-03-11T19:56:01.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/693: link d1/l8 d1/d10/d13/d6c/d2d/d4d/d91/dcc/ld6 0 2024-03-11T19:56:01.767 INFO:tasks.workunit.client.0.smithi107.stdout:9/627: rename d4/d9/dae/d6d/f2d to d4/d1d/d5c/fde 0 2024-03-11T19:56:01.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/628: creat d4/d1d/d1f/d28/d2c/d96/dd3/fdf x:0 0 0 2024-03-11T19:56:01.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/717: mknod da/d14/d96/dc1/cdb 0 2024-03-11T19:56:01.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/629: creat d4/d1d/d5c/d97/fe0 x:0 0 0 2024-03-11T19:56:01.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/625: getdents d4/d7/d1f/d2b/d19/d75/d8e/da5 0 2024-03-11T19:56:01.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/626: fsync d4/d7/d1f/d2b/d19/f70 0 2024-03-11T19:56:01.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/627: write d4/d7/d1f/d82/f83 [16995,14309] 0 2024-03-11T19:56:01.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/694: creat d1/d10/d21/d43/fd7 x:0 0 0 2024-03-11T19:56:01.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/695: rmdir d1/d10/d2a 39 2024-03-11T19:56:01.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/718: rmdir da/d31/d42 39 2024-03-11T19:56:01.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/719: truncate da/d14/d18/d1e/d9b/fb9 632986 0 2024-03-11T19:56:01.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/720: write da/d14/d43/d47/fc7 [639037,40888] 0 2024-03-11T19:56:01.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/721: write da/d14/d43/fce [811086,50891] 0 2024-03-11T19:56:01.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/696: creat d1/d10/d13/d6c/d2d/d4d/d91/db4/fd8 x:0 0 0 2024-03-11T19:56:01.778 INFO:tasks.workunit.client.0.smithi107.stdout:4/628: creat d4/d7/d2f/d3c/fc6 x:0 0 0 2024-03-11T19:56:01.778 INFO:tasks.workunit.client.0.smithi107.stdout:5/722: creat da/d14/d96/dc1/fdc x:0 0 0 2024-03-11T19:56:01.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/697: rmdir d1/d10/d13 39 2024-03-11T19:56:01.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/629: mknod d4/d7/d1f/d2b/d24/d26/d29/dc5/cc7 0 2024-03-11T19:56:01.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/630: read - d4/d7/da0/fb9 zero size 2024-03-11T19:56:01.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/631: chown d4/d7/db/c12 4409 1 2024-03-11T19:56:01.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/626: dwrite de/d11/d23/d24/f69 [0,4194304] 0 2024-03-11T19:56:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:4/632: dread d4/d7/d1f/d2b/d24/f5b [0,4194304] 0 2024-03-11T19:56:01.797 INFO:tasks.workunit.client.0.smithi107.stdout:9/630: write d4/d1d/d1f/d28/d2c/f5b [4442767,494] 0 2024-03-11T19:56:01.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/698: symlink d1/d10/d4e/d73/d8d/d9c/ld9 0 2024-03-11T19:56:01.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/699: chown d1/cf 12889 1 2024-03-11T19:56:01.800 INFO:tasks.workunit.client.0.smithi107.stdout:1/677: dwrite d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/f65 [0,4194304] 0 2024-03-11T19:56:01.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/633: creat d4/d7/db/db8/fc8 x:0 0 0 2024-03-11T19:56:01.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/678: creat d6/d86/dc0/fda x:0 0 0 2024-03-11T19:56:01.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/631: creat d4/d1d/d1f/d28/d5d/fe1 x:0 0 0 2024-03-11T19:56:01.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/700: creat d1/d10/d21/fda x:0 0 0 2024-03-11T19:56:01.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/701: write d1/d45/f86 [5175976,64729] 0 2024-03-11T19:56:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/634: rename d4/d7/d1f/d2b/d19/d9a/fc1 to d4/fc9 0 2024-03-11T19:56:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/635: truncate d4/d7/d1f/d2b/d24/d26/fae 4721089 0 2024-03-11T19:56:01.830 INFO:tasks.workunit.client.0.smithi107.stdout:4/636: dread d4/d7/d1f/d2b/d24/d26/f84 [0,4194304] 0 2024-03-11T19:56:01.831 INFO:tasks.workunit.client.0.smithi107.stdout:4/637: mkdir d4/d7/d2f/d33/d7b/dca 0 2024-03-11T19:56:01.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/627: write de/d11/d23/d24/f69 [2868756,8351] 0 2024-03-11T19:56:01.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/628: write de/d11/f2c [5062801,45188] 0 2024-03-11T19:56:01.848 INFO:tasks.workunit.client.0.smithi107.stdout:6/582: dwrite d10/d16/d24/d2e/d9d/f92 [0,4194304] 0 2024-03-11T19:56:01.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/723: dread da/d14/f68 [0,4194304] 0 2024-03-11T19:56:01.852 INFO:tasks.workunit.client.0.smithi107.stdout:4/638: mkdir d4/d7/da0/dcb 0 2024-03-11T19:56:01.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/702: read d1/f9d [269756,3703] 0 2024-03-11T19:56:01.862 INFO:tasks.workunit.client.0.smithi107.stdout:3/703: unlink d1/d10/d4e/d73/d8d/lac 0 2024-03-11T19:56:01.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/704: creat d1/d10/d26/fdb x:0 0 0 2024-03-11T19:56:01.865 INFO:tasks.workunit.client.0.smithi107.stdout:3/705: symlink d1/d10/d13/d6c/d2d/d4d/d91/da5/ldc 0 2024-03-11T19:56:01.865 INFO:tasks.workunit.client.0.smithi107.stdout:1/679: dwrite d6/f14 [0,4194304] 0 2024-03-11T19:56:01.866 INFO:tasks.workunit.client.0.smithi107.stdout:1/680: symlink d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82/ldb 0 2024-03-11T19:56:01.867 INFO:tasks.workunit.client.0.smithi107.stdout:1/681: symlink d6/d7e/d81/d27/d2c/d72/dc3/ldc 0 2024-03-11T19:56:01.868 INFO:tasks.workunit.client.0.smithi107.stdout:4/639: getdents d4/d7/d2f/d33/d3a 0 2024-03-11T19:56:01.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/682: unlink d6/f16 0 2024-03-11T19:56:01.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/683: unlink d6/d86/fcb 0 2024-03-11T19:56:01.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/684: link d6/lf d6/d7e/d81/d27/d2c/d32/ldd 0 2024-03-11T19:56:01.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/583: dwrite d10/d16/d24/d2e/d9d/fa8 [0,4194304] 0 2024-03-11T19:56:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/584: symlink d10/d16/d2b/d39/lc1 0 2024-03-11T19:56:01.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/640: dread d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:56:01.902 INFO:tasks.workunit.client.0.smithi107.stdout:4/641: creat d4/d7/db/d5e/fcc x:0 0 0 2024-03-11T19:56:01.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/685: dwrite d6/d86/dc0/fda [0,4194304] 0 2024-03-11T19:56:01.924 INFO:tasks.workunit.client.0.smithi107.stdout:1/686: mknod d6/d7e/d81/d27/d2c/d32/dc1/dc9/cde 0 2024-03-11T19:56:01.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/642: dwrite d4/d7/da0/fba [0,4194304] 0 2024-03-11T19:56:01.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/643: fdatasync d4/d7/d1f/d2b/d24/d26/d29/f64 0 2024-03-11T19:56:01.940 INFO:tasks.workunit.client.0.smithi107.stdout:4/644: creat d4/d7/d1f/d2b/d19/d9a/fcd x:0 0 0 2024-03-11T19:56:01.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/645: read d4/d7/d1f/d2b/d19/f2a [1426722,92327] 0 2024-03-11T19:56:01.957 INFO:tasks.workunit.client.0.smithi107.stdout:4/646: mknod d4/d7/d2f/d3b/cce 0 2024-03-11T19:56:01.967 INFO:tasks.workunit.client.0.smithi107.stdout:1/687: dwrite d6/d7e/d81/d27/d2c/d32/d58/fc8 [0,4194304] 0 2024-03-11T19:56:02.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/647: dread d4/fc [0,4194304] 0 2024-03-11T19:56:02.013 INFO:tasks.workunit.client.0.smithi107.stdout:4/648: dread d4/d7/d1f/d2b/d19/f2a [0,4194304] 0 2024-03-11T19:56:02.015 INFO:tasks.workunit.client.0.smithi107.stdout:4/649: symlink d4/d7/d1f/d2b/d24/lcf 0 2024-03-11T19:56:02.015 INFO:tasks.workunit.client.0.smithi107.stdout:1/688: dwrite d6/d7e/d81/d27/f3e [0,4194304] 0 2024-03-11T19:56:02.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/689: dwrite d6/d28/dc4/f91 [0,4194304] 0 2024-03-11T19:56:02.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/690: symlink d6/d7e/da5/ldf 0 2024-03-11T19:56:02.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/691: stat d6/d7e/d81/d27/d2c/l71 0 2024-03-11T19:56:03.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:02 smithi122 ceph-mon[36397]: pgmap v81: 129 pgs: 129 active+clean; 1.4 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s rd, 213 MiB/s wr, 757 op/s 2024-03-11T19:56:03.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:02 smithi145 ceph-mon[36730]: pgmap v81: 129 pgs: 129 active+clean; 1.4 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s rd, 213 MiB/s wr, 757 op/s 2024-03-11T19:56:03.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:02 smithi107 ceph-mon[31712]: pgmap v81: 129 pgs: 129 active+clean; 1.4 GiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 170 MiB/s rd, 213 MiB/s wr, 757 op/s 2024-03-11T19:56:05.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:04 smithi122 ceph-mon[36397]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 197 MiB/s wr, 540 op/s 2024-03-11T19:56:05.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:04 smithi145 ceph-mon[36730]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 197 MiB/s wr, 540 op/s 2024-03-11T19:56:05.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:04 smithi107 ceph-mon[31712]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 197 MiB/s wr, 540 op/s 2024-03-11T19:56:07.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:06 smithi122 ceph-mon[36397]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 184 MiB/s wr, 480 op/s 2024-03-11T19:56:07.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:06 smithi145 ceph-mon[36730]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 184 MiB/s wr, 480 op/s 2024-03-11T19:56:07.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:06 smithi107 ceph-mon[31712]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 141 MiB/s rd, 184 MiB/s wr, 480 op/s 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.920+0000 7f26dfe00640 1 -- 172.21.15.107:0/4175711198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d81035d0 msgr2=0x7f26d81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.920+0000 7f26dfe00640 1 --2- 172.21.15.107:0/4175711198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d81035d0 0x7f26d81039b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009be0 tx=0x7f26cc02bef0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 -- 172.21.15.107:0/4175711198 shutdown_connections 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 --2- 172.21.15.107:0/4175711198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 0x7f26d8110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 --2- 172.21.15.107:0/4175711198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d8103f80 0x7f26d8107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 --2- 172.21.15.107:0/4175711198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d81035d0 0x7f26d81039b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 -- 172.21.15.107:0/4175711198 >> 172.21.15.107:0/4175711198 conn(0x7f26d80fee30 msgr2=0x7f26d8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 -- 172.21.15.107:0/4175711198 shutdown_connections 2024-03-11T19:56:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.921+0000 7f26dfe00640 1 -- 172.21.15.107:0/4175711198 wait complete. 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.922+0000 7f26dfe00640 1 Processor -- start 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.922+0000 7f26dfe00640 1 -- start start 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.922+0000 7f26dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.922+0000 7f26dfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 0x7f26d819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.922+0000 7f26dfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d8199770 0x7f26d8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dfe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819a240 con 0x7f26d81035d0 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dfe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819a3b0 con 0x7f26d81086a0 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dfe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819a570 con 0x7f26d8199770 2024-03-11T19:56:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57614/0 (socket says 172.21.15.107:57614) 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 -- 172.21.15.107:0/2572709027 learned_addr learned my addr 172.21.15.107:0/2572709027 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dde00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d8199770 0x7f26d8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dca00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 0x7f26d819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 -- 172.21.15.107:0/2572709027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d8199770 msgr2=0x7f26d8199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d8199770 0x7f26d8199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 -- 172.21.15.107:0/2572709027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 msgr2=0x7f26d819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 0x7f26d819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 -- 172.21.15.107:0/2572709027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009560 con 0x7f26d81035d0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.923+0000 7f26dd400640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26cc036040 tx=0x7f26cc033610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.924+0000 7f26d6000640 1 -- 172.21.15.107:0/2572709027 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc003d80 con 0x7f26d81035d0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.924+0000 7f26d6000640 1 -- 172.21.15.107:0/2572709027 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc02ce40 con 0x7f26d81035d0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.924+0000 7f26d6000640 1 -- 172.21.15.107:0/2572709027 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc034990 con 0x7f26d81035d0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.924+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc009840 con 0x7f26d81035d0 2024-03-11T19:56:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.924+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d81a6700 con 0x7f26d81035d0 2024-03-11T19:56:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.927+0000 7f26d6000640 1 -- 172.21.15.107:0/2572709027 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f26cc004640 con 0x7f26d81035d0 2024-03-11T19:56:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.927+0000 7f26e2c35740 1 Processor -- start 2024-03-11T19:56:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.927+0000 7f26e2c35740 1 -- start start 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a33db0 0x558b67a34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a3d330 0x558b67a3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b679ef620 con 0x558b67a34b20 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b679a18c0 con 0x558b67a3d330 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26e2c35740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b6799edf0 con 0x558b67a33db0 2024-03-11T19:56:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57624/0 (socket says 172.21.15.107:57624) 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 -- 172.21.15.107:0/1111447704 learned_addr learned my addr 172.21.15.107:0/1111447704 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dd400640 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a33db0 0x558b67a34230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dca00640 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a3d330 0x558b67a3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 -- 172.21.15.107:0/1111447704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a33db0 msgr2=0x558b67a34230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a33db0 0x558b67a34230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 -- 172.21.15.107:0/1111447704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a3d330 msgr2=0x558b67a3f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a3d330 0x558b67a3f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.928+0000 7f26dde00640 1 -- 172.21.15.107:0/1111447704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b6791bb00 con 0x558b67a34b20 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26d6000640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 0x7f26c00a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26dde00640 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009c20 tx=0x7f26d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e7d0c04cd52d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26d6000640 1 -- 172.21.15.107:0/2572709027 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f26cc0f0110 con 0x7f26d81035d0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26dd400640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 0x7f26c00a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26ca600640 1 -- 172.21.15.107:0/1111447704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0019630 con 0x558b67a34b20 2024-03-11T19:56:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26ca600640 1 -- 172.21.15.107:0/1111447704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0009e90 con 0x558b67a34b20 2024-03-11T19:56:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.929+0000 7f26ca600640 1 -- 172.21.15.107:0/1111447704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000eb00 con 0x558b67a34b20 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 -- 172.21.15.107:0/1111447704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 msgr2=0x558b67a33870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26dd400640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 0x7f26c00a4100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x558b67a35280 tx=0x7f26cc0f3b80 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009c20 tx=0x7f26d0011810 comp rx=0 tx=0).stop 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 -- 172.21.15.107:0/1111447704 shutdown_connections 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a3d330 0x558b67a3f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a33db0 0x558b67a34230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 --2- 172.21.15.107:0/1111447704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a33870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 -- 172.21.15.107:0/1111447704 >> 172.21.15.107:0/1111447704 conn(0x558b679fb4b0 msgr2=0x558b67a2eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 -- 172.21.15.107:0/1111447704 shutdown_connections 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.930+0000 7f26e2c35740 1 -- 172.21.15.107:0/1111447704 wait complete. 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 -- start start 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 Processor -- start 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a34b20 0x558b67a3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67a3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b6799a5d0 con 0x558b67a33db0 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b6793fb50 con 0x558b67a34b20 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b6793f7f0 con 0x558b67a3d330 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.931+0000 7f26e2c35740 10 client.0 _ll_register_callbacks cb 0x558b67953550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57638/0 (socket says 172.21.15.107:57638) 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 -- 172.21.15.107:0/3566928972 learned_addr learned my addr 172.21.15.107:0/3566928972 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dca00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a34b20 0x558b67a3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67a3adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 -- 172.21.15.107:0/3566928972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 msgr2=0x558b67a3adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67a3adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 -- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a34b20 msgr2=0x558b67a3a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a34b20 0x558b67a3a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 -- 172.21.15.107:0/3566928972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d00098d0 con 0x558b67a33db0 2024-03-11T19:56:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26dde00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f26d0005890 tx=0x7f26d0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924b2e06c24e16d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26bea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.932+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d002b070 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f26d000ea50 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b6796ccd0 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d000ec10 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 10 client.15597 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b67998c50 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b6796ccd0 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f26d002e020 con 0x558b67a33db0 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 10 client.15597 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 20 client.15597 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67a4d330 0x558b67a4f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 4 client.15597 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:56:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.933+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b67990ce0 con 0x558b67a4d330 2024-03-11T19:56:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.934+0000 7f26dca00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67a4d330 0x558b67a4f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.934+0000 7f26dca00640 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67a4d330 0x558b67a4f750 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934d6208ad8b1298 server_cookie=5ef95fc110171a74 in_seq=0 out_seq=0 2024-03-11T19:56:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.934+0000 7f26bea00640 10 client.15597 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26bea00640 1 -- 172.21.15.107:0/3566928972 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b67990ce0 con 0x558b67a4d330 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26bea00640 10 client.15597 handle_command_reply: tid=0 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 client.15597 shutdown 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 msgr2=0x558b67a3a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f26d0005890 tx=0x7f26d0009680 comp rx=0 tx=0).stop 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 shutdown_connections 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67a4d330 0x558b67a4f750 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a34b20 0x558b67a3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67a3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3566928972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a33db0 0x558b67a3a370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 >> 172.21.15.107:0/3566928972 conn(0x558b679fb4b0 msgr2=0x558b67a307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.935+0000 7f26bea00640 0 client.15597 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.936+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 shutdown_connections 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.936+0000 7f26e2c35740 1 -- 172.21.15.107:0/3566928972 wait complete. 2024-03-11T19:56:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.936+0000 7f26e2c35740 20 client.15597 trim_cache size 0 max 16384 2024-03-11T19:56:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.950+0000 7f26e2c35740 1 Processor -- start 2024-03-11T19:56:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.950+0000 7f26e2c35740 1 -- start start 2024-03-11T19:56:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a3caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a34b20 0x558b67a35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b676e2e60 con 0x558b67a3d330 2024-03-11T19:56:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b67a2faa0 con 0x558b67a33db0 2024-03-11T19:56:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26e2c35740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b67a206d0 con 0x558b67a34b20 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a3caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57652/0 (socket says 172.21.15.107:57652) 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 -- 172.21.15.107:0/2286343923 learned_addr learned my addr 172.21.15.107:0/2286343923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dca00640 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a34b20 0x558b67a35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 -- 172.21.15.107:0/2286343923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a34b20 msgr2=0x558b67a35690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a34b20 0x558b67a35690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 -- 172.21.15.107:0/2286343923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 msgr2=0x558b67a3caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a3caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.951+0000 7f26dde00640 1 -- 172.21.15.107:0/2286343923 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b67935530 con 0x558b67a3d330 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26dde00640 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009c20 tx=0x7f26d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd77745ee51fc81 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26c9c00640 1 -- 172.21.15.107:0/2286343923 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d002b040 con 0x558b67a3d330 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26c9c00640 1 -- 172.21.15.107:0/2286343923 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0011d10 con 0x558b67a3d330 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26c9c00640 1 -- 172.21.15.107:0/2286343923 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004680 con 0x558b67a3d330 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26e2c35740 1 -- 172.21.15.107:0/2286343923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 msgr2=0x558b67a35bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.952+0000 7f26e2c35740 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009c20 tx=0x7f26d0000bc0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 -- 172.21.15.107:0/2286343923 shutdown_connections 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a3d330 0x558b67a35bd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a34b20 0x558b67a35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 --2- 172.21.15.107:0/2286343923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a3caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 -- 172.21.15.107:0/2286343923 >> 172.21.15.107:0/2286343923 conn(0x558b679fb4b0 msgr2=0x558b67a54070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 -- 172.21.15.107:0/2286343923 shutdown_connections 2024-03-11T19:56:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 -- 172.21.15.107:0/2286343923 wait complete. 2024-03-11T19:56:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 -- start start 2024-03-11T19:56:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.953+0000 7f26e2c35740 1 Processor -- start 2024-03-11T19:56:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67aa27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b67a42fa0 con 0x558b67a34b20 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b67aa2d60 con 0x558b67a33db0 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b67aa2f20 con 0x558b67a3d330 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26e2c35740 10 client.0 _ll_register_callbacks cb 0x7f26d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57666/0 (socket says 172.21.15.107:57666) 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 -- 172.21.15.107:0/3263761496 learned_addr learned my addr 172.21.15.107:0/3263761496 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dde00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67aa27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dde00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 -- 172.21.15.107:0/3263761496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 msgr2=0x558b67aa27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67aa27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 -- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 msgr2=0x558b67a422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dca00640 1 -- 172.21.15.107:0/3263761496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b67998c50 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.954+0000 7f26dde00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26dca00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26d819af70 tx=0x7f26c4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9aef789f5429bab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26bea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4018040 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b679bfd30 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f26c401d370 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c401d530 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26e2c35740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.955+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b6795b2c0 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b679bfd30 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f26c4025410 con 0x558b67a34b20 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 20 client.15603 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67aa47c0 0x558b67aa6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 4 client.15603 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b6769a600 con 0x558b67aa47c0 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.956+0000 7f26dde00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67aa47c0 0x558b67aa6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.957+0000 7f26dde00640 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67aa47c0 0x558b67aa6be0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552b2095b0258b52 server_cookie=f31c403f3b0edc5d in_seq=0 out_seq=0 2024-03-11T19:56:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:07.957+0000 7f26bea00640 10 client.15603 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:08.837 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:08 smithi145 ceph-mon[36730]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s rd, 191 MiB/s wr, 514 op/s 2024-03-11T19:56:09.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:08 smithi122 ceph-mon[36397]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s rd, 191 MiB/s wr, 514 op/s 2024-03-11T19:56:09.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:08 smithi107 ceph-mon[31712]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 148 MiB/s rd, 191 MiB/s wr, 514 op/s 2024-03-11T19:56:11.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:10 smithi122 ceph-mon[36397]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 195 op/s 2024-03-11T19:56:11.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:10 smithi145 ceph-mon[36730]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 195 op/s 2024-03-11T19:56:11.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:10 smithi107 ceph-mon[31712]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 195 op/s 2024-03-11T19:56:13.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:12 smithi122 ceph-mon[36397]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 353 op/s 2024-03-11T19:56:13.035 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:12 smithi145 ceph-mon[36730]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 353 op/s 2024-03-11T19:56:13.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:12 smithi107 ceph-mon[31712]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 101 MiB/s wr, 353 op/s 2024-03-11T19:56:15.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:14 smithi122 ceph-mon[36397]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 33 MiB/s wr, 283 op/s 2024-03-11T19:56:15.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:14 smithi145 ceph-mon[36730]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 33 MiB/s wr, 283 op/s 2024-03-11T19:56:15.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:14 smithi107 ceph-mon[31712]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 33 MiB/s wr, 283 op/s 2024-03-11T19:56:17.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:16 smithi122 ceph-mon[36397]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 248 op/s 2024-03-11T19:56:17.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:16 smithi145 ceph-mon[36730]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 248 op/s 2024-03-11T19:56:17.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:16 smithi107 ceph-mon[31712]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 248 op/s 2024-03-11T19:56:18.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/609: sync 2024-03-11T19:56:18.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/547: sync 2024-03-11T19:56:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/692: sync 2024-03-11T19:56:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/693: write d6/d19/f43 [4960038,13009] 0 2024-03-11T19:56:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/694: fdatasync d6/d86/fbc 0 2024-03-11T19:56:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:9/632: rename d4/d1d/d5c/fc7 to d4/d1d/d5c/d97/fe2 0 2024-03-11T19:56:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/695: truncate d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3/fba 67902 0 2024-03-11T19:56:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.264+0000 7f26bea00640 1 -- 172.21.15.107:0/3263761496 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b6769a600 con 0x558b67aa47c0 2024-03-11T19:56:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.264+0000 7f26bea00640 10 client.15603 handle_command_reply: tid=0 2024-03-11T19:56:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.264+0000 7f26e2c35740 1 client.15603 shutdown 2024-03-11T19:56:18.267 INFO:tasks.workunit.client.0.smithi107.stdout:1/696: write d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/f8f [264888,124859] 0 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 msgr2=0x558b67a42800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26d819af70 tx=0x7f26c4009f80 comp rx=0 tx=0).stop 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 shutdown_connections 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558b67aa47c0 0x558b67aa6be0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558b67a3d330 0x558b67aa27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b67a34b20 0x558b67a42800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 --2- 172.21.15.107:0/3263761496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b67a33db0 0x558b67a422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26bea00640 0 client.15603 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:18.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.265+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 >> 172.21.15.107:0/3263761496 conn(0x558b679fb4b0 msgr2=0x558b67a35690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:18.269 INFO:tasks.workunit.client.0.smithi107.stdout:5/724: rename da/d14/d18/f4c to da/d14/d38/d59/dbd/fdd 0 2024-03-11T19:56:18.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/697: mknod d6/d19/ce0 0 2024-03-11T19:56:18.269 INFO:tasks.workunit.client.0.smithi107.stdout:7/743: sync 2024-03-11T19:56:18.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/629: sync 2024-03-11T19:56:18.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/630: read - de/d58/fa7 zero size 2024-03-11T19:56:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.268+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 shutdown_connections 2024-03-11T19:56:18.271 INFO:tasks.workunit.client.0.smithi107.stdout:0/631: chown de/d11/d23/d32/d34/d39/d3a/f50 10 1 2024-03-11T19:56:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.268+0000 7f26e2c35740 1 -- 172.21.15.107:0/3263761496 wait complete. 2024-03-11T19:56:18.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/633: creat d4/d9/dae/fe3 x:0 0 0 2024-03-11T19:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.269+0000 7f26e2c35740 20 client.15603 trim_cache size 0 max 16384 2024-03-11T19:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 msgr2=0x7f26c00a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 0x7f26c00a4100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x558b67a35280 tx=0x7f26cc0f3b80 comp rx=0 tx=0).stop 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 msgr2=0x7f26d819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26cc036040 tx=0x7f26cc033610 comp rx=0 tx=0).stop 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 shutdown_connections 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26c00a1c40 0x7f26c00a4100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81086a0 0x7f26d819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26d8199770 0x7f26d8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 --2- 172.21.15.107:0/2572709027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d81035d0 0x7f26d819f660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.270+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 >> 172.21.15.107:0/2572709027 conn(0x7f26d80fee30 msgr2=0x7f26d80ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.271+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 shutdown_connections 2024-03-11T19:56:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:18.271+0000 7f26dfe00640 1 -- 172.21.15.107:0/2572709027 wait complete. 2024-03-11T19:56:18.275 INFO:tasks.workunit.client.0.smithi107.stdout:3/706: rename d1/d10/d21/d43/f84 to d1/d45/fdd 0 2024-03-11T19:56:18.275 INFO:tasks.workunit.client.0.smithi107.stdout:3/707: fdatasync d1/d10/d21/d43/d48/d98/f9e 0 2024-03-11T19:56:18.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/698: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/fe1 x:0 0 0 2024-03-11T19:56:18.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/725: creat da/d14/d38/d59/dbd/fde x:0 0 0 2024-03-11T19:56:18.278 INFO:tasks.workunit.client.0.smithi107.stdout:9/634: creat d4/d9/d9f/dd8/fe4 x:0 0 0 2024-03-11T19:56:18.281 INFO:tasks.workunit.client.0.smithi107.stdout:3/708: unlink d1/d10/d13/d6c/d2d/d4d/d91/da5/ldc 0 2024-03-11T19:56:18.282 INFO:tasks.workunit.client.0.smithi107.stdout:6/585: rename d10/d19/l52 to d10/lc2 0 2024-03-11T19:56:18.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/635: creat d4/d1d/d1f/d28/d2c/d96/fe5 x:0 0 0 2024-03-11T19:56:18.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/726: dread da/d14/d18/d1e/d9b/d73/fa3 [0,4194304] 0 2024-03-11T19:56:18.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/699: dread d6/d7e/d81/d27/d2c/d32/f7f [0,4194304] 0 2024-03-11T19:56:18.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/700: creat d6/d86/dc0/fe2 x:0 0 0 2024-03-11T19:56:18.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/632: dread de/d11/d23/d24/da4/fbd [0,4194304] 0 2024-03-11T19:56:18.313 INFO:tasks.workunit.client.0.smithi107.stdout:0/633: truncate de/d11/d74/fa5 342744 0 2024-03-11T19:56:18.313 INFO:tasks.workunit.client.0.smithi107.stdout:0/634: readlink de/d11/l77 0 2024-03-11T19:56:18.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/650: rename d4/d7/d2f/d33/d3a to d4/d7/d1f/d2b/d93/dc3/dd0 0 2024-03-11T19:56:18.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/635: symlink de/d11/d23/d24/da4/dae/lbe 0 2024-03-11T19:56:18.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/709: rename d1/f6e to d1/d10/d4e/fde 0 2024-03-11T19:56:18.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/651: unlink d4/d7/d1f/d2b/d19/l74 0 2024-03-11T19:56:18.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/636: mkdir de/d11/d23/d26/dbf 0 2024-03-11T19:56:18.320 INFO:tasks.workunit.client.0.smithi107.stdout:4/652: mkdir d4/d7/d2f/d33/d7b/dca/dd1 0 2024-03-11T19:56:18.321 INFO:tasks.workunit.client.0.smithi107.stdout:6/586: dread d10/d16/d1f/f79 [0,4194304] 0 2024-03-11T19:56:18.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/653: rename d4/d7/da0/laf to d4/d7/d2f/d33/d7b/ld2 0 2024-03-11T19:56:18.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/587: symlink d10/d16/d24/d2e/lc3 0 2024-03-11T19:56:18.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/654: rmdir d4/d7/d1f/d82 39 2024-03-11T19:56:18.340 INFO:tasks.workunit.client.0.smithi107.stdout:5/727: dwrite da/d14/d18/d1e/d9b/fa4 [0,4194304] 0 2024-03-11T19:56:18.341 INFO:tasks.workunit.client.0.smithi107.stdout:5/728: chown f5 3926695 1 2024-03-11T19:56:18.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/636: dwrite d4/d9/d4d/fd6 [0,4194304] 0 2024-03-11T19:56:18.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/637: dread - d4/d1c/d37/f75 zero size 2024-03-11T19:56:18.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/638: chown d4/d1d/d1f/d28/f42 45027245 1 2024-03-11T19:56:18.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/639: chown d4/fe 31289 1 2024-03-11T19:56:18.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/640: write d4/d1d/d1f/d28/d2c/d96/fe5 [543465,118918] 0 2024-03-11T19:56:18.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/710: write d1/d10/d21/d43/d48/f50 [946784,121721] 0 2024-03-11T19:56:18.372 INFO:tasks.workunit.client.0.smithi107.stdout:9/641: write d4/d1d/d5c/d64/f9d [7706484,10595] 0 2024-03-11T19:56:18.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/637: dwrite de/d11/d74/fa5 [0,4194304] 0 2024-03-11T19:56:18.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/638: chown de/d11/fb0 92605652 1 2024-03-11T19:56:18.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/711: dread d1/d10/d13/d6c/f32 [0,4194304] 0 2024-03-11T19:56:18.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/712: readlink d1/d10/d13/d6c/d2d/d4d/d91/dcc/ld6 0 2024-03-11T19:56:18.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/713: truncate d1/d10/d13/f2c 4844444 0 2024-03-11T19:56:18.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/655: dread d4/d7/d2f/f65 [0,4194304] 0 2024-03-11T19:56:18.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/656: creat d4/d7/d1f/d2b/d19/d9a/fd3 x:0 0 0 2024-03-11T19:56:18.403 INFO:tasks.workunit.client.0.smithi107.stdout:3/714: dread d1/d10/d26/f58 [0,4194304] 0 2024-03-11T19:56:18.403 INFO:tasks.workunit.client.0.smithi107.stdout:3/715: stat d1/d10/d4e/d73/d8d/d9c/cb7 0 2024-03-11T19:56:18.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/716: rename d1/d10/d13/d6c/fd4 to d1/d10/d13/d6c/fdf 0 2024-03-11T19:56:18.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/717: mknod d1/d10/d13/d4a/d4c/ce0 0 2024-03-11T19:56:18.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/729: dwrite da/d14/d43/d47/fc7 [0,4194304] 0 2024-03-11T19:56:18.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/718: getdents d1/d10/d2a/d7e/d8b 0 2024-03-11T19:56:18.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/719: truncate d1/d10/d26/d17/f81 1435489 0 2024-03-11T19:56:18.411 INFO:tasks.workunit.client.0.smithi107.stdout:6/588: dwrite d10/d16/d1f/d51/d76/f88 [0,4194304] 0 2024-03-11T19:56:18.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/730: read da/f85 [183047,101615] 0 2024-03-11T19:56:18.412 INFO:tasks.workunit.client.0.smithi107.stdout:5/731: fsync da/d14/d18/d1e/d9b/fb9 0 2024-03-11T19:56:18.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/720: link d1/d10/f57 d1/d10/d2a/d7e/d8b/fe1 0 2024-03-11T19:56:18.413 INFO:tasks.workunit.client.0.smithi107.stdout:3/721: dread - d1/d10/d26/fdb zero size 2024-03-11T19:56:18.413 INFO:tasks.workunit.client.0.smithi107.stdout:3/722: fsync d1/d10/d13/f61 0 2024-03-11T19:56:18.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/732: creat da/d14/d18/fdf x:0 0 0 2024-03-11T19:56:18.415 INFO:tasks.workunit.client.0.smithi107.stdout:6/589: creat d10/d16/d2b/da5/fc4 x:0 0 0 2024-03-11T19:56:18.416 INFO:tasks.workunit.client.0.smithi107.stdout:3/723: creat d1/d10/d26/d17/fe2 x:0 0 0 2024-03-11T19:56:18.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/642: dwrite d4/d1c/d37/fbb [4194304,4194304] 0 2024-03-11T19:56:18.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/724: dread d1/d10/d13/d6c/d2d/d4d/f53 [0,4194304] 0 2024-03-11T19:56:18.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/725: chown d1/d10/d13/d6c/d2d/d4d/d91/da5/cd2 523484 1 2024-03-11T19:56:18.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/726: mkdir d1/dbb/de3 0 2024-03-11T19:56:18.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/727: creat d1/dbb/fe4 x:0 0 0 2024-03-11T19:56:18.436 INFO:tasks.workunit.client.0.smithi107.stdout:6/590: rename lb to d10/d16/d2b/d39/d6a/d6e/lc5 0 2024-03-11T19:56:18.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/639: dwrite de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:56:18.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/591: mknod d10/d16/d1f/cc6 0 2024-03-11T19:56:18.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/592: write d10/d19/f5d [423036,32047] 0 2024-03-11T19:56:18.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/593: unlink d10/f53 0 2024-03-11T19:56:18.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/594: rmdir d10/d16/d2b/d39/d6a 39 2024-03-11T19:56:18.459 INFO:tasks.workunit.client.0.smithi107.stdout:6/595: creat d10/d16/d1f/d51/d7a/dba/fc7 x:0 0 0 2024-03-11T19:56:18.462 INFO:tasks.workunit.client.0.smithi107.stdout:6/596: symlink d10/d16/d2b/d39/lc8 0 2024-03-11T19:56:18.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/733: dwrite da/d14/d18/d1e/d9b/d73/fd6 [0,4194304] 0 2024-03-11T19:56:18.468 INFO:tasks.workunit.client.0.smithi107.stdout:9/643: dwrite d4/d1d/d1f/d28/d5d/f7f [0,4194304] 0 2024-03-11T19:56:18.476 INFO:tasks.workunit.client.0.smithi107.stdout:0/640: read de/d11/d23/d32/f54 [3426885,44318] 0 2024-03-11T19:56:18.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/734: mknod da/d31/ce0 0 2024-03-11T19:56:18.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/641: mknod de/d11/d74/da3/cc0 0 2024-03-11T19:56:18.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/642: readlink de/d11/d23/d24/lb5 0 2024-03-11T19:56:18.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/643: creat de/d11/d23/d32/d34/d39/d3a/d72/fc1 x:0 0 0 2024-03-11T19:56:18.482 INFO:tasks.workunit.client.0.smithi107.stdout:3/728: dwrite d1/d10/d13/d4a/d4c/d49/fb5 [0,4194304] 0 2024-03-11T19:56:18.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/644: symlink de/d11/d23/d32/d34/d39/d3a/d72/lc2 0 2024-03-11T19:56:18.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/645: write de/d11/d23/d24/f60 [724379,23964] 0 2024-03-11T19:56:18.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/646: unlink de/d7a/f97 0 2024-03-11T19:56:18.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/597: write d10/d16/d24/f44 [1008206,32213] 0 2024-03-11T19:56:18.492 INFO:tasks.workunit.client.0.smithi107.stdout:6/598: getdents d10/d16/d2b/da5 0 2024-03-11T19:56:18.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/647: symlink de/d11/d23/d32/d34/d39/d3a/lc3 0 2024-03-11T19:56:18.495 INFO:tasks.workunit.client.0.smithi107.stdout:0/648: mkdir de/d11/d23/d24/d83/dc4 0 2024-03-11T19:56:18.510 INFO:tasks.workunit.client.0.smithi107.stdout:3/729: read d1/d10/d2a/d7e/d8b/f97 [2987064,23130] 0 2024-03-11T19:56:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/730: rmdir d1/d10/d21/d43 39 2024-03-11T19:56:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:6/599: truncate d10/d19/f74 4794131 0 2024-03-11T19:56:18.523 INFO:tasks.workunit.client.0.smithi107.stdout:0/649: rename de/d11/d23/d32/d34/c47 to de/d11/d23/d24/d4f/d89/cc5 0 2024-03-11T19:56:18.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/735: dwrite da/d14/d18/d1e/d9b/d73/fa3 [0,4194304] 0 2024-03-11T19:56:18.535 INFO:tasks.workunit.client.0.smithi107.stdout:6/600: rename d10/d16/d24/f44 to d10/d3d/d54/fc9 0 2024-03-11T19:56:18.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/601: mkdir d10/d16/d2b/d39/dca 0 2024-03-11T19:56:18.543 INFO:tasks.workunit.client.0.smithi107.stdout:9/644: dwrite d4/d1c/f9e [0,4194304] 0 2024-03-11T19:56:18.566 INFO:tasks.workunit.client.0.smithi107.stdout:0/650: dread de/f3d [0,4194304] 0 2024-03-11T19:56:18.567 INFO:tasks.workunit.client.0.smithi107.stdout:0/651: mkdir de/d11/d23/d24/dc6 0 2024-03-11T19:56:18.567 INFO:tasks.workunit.client.0.smithi107.stdout:0/652: write de/d11/d23/d32/d34/d39/d3a/d72/fc1 [104374,8598] 0 2024-03-11T19:56:18.567 INFO:tasks.workunit.client.0.smithi107.stdout:0/653: chown de/d11/d23/c37 685699 1 2024-03-11T19:56:18.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/654: creat de/d11/d23/d24/d4f/fc7 x:0 0 0 2024-03-11T19:56:18.570 INFO:tasks.workunit.client.0.smithi107.stdout:0/655: rmdir de 39 2024-03-11T19:56:18.570 INFO:tasks.workunit.client.0.smithi107.stdout:0/656: write de/f88 [1693569,25058] 0 2024-03-11T19:56:18.570 INFO:tasks.workunit.client.0.smithi107.stdout:0/657: dread - de/d11/d23/d32/d34/d39/d3a/d72/fb2 zero size 2024-03-11T19:56:18.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/602: write d10/d19/f77 [1142830,87880] 0 2024-03-11T19:56:18.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/603: dread - d10/d19/d3b/f5e zero size 2024-03-11T19:56:18.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/731: dwrite d1/d10/d13/d4a/d4c/f2f [8388608,4194304] 0 2024-03-11T19:56:18.572 INFO:tasks.workunit.client.0.smithi107.stdout:9/645: write d4/d1d/d1f/d28/d2c/f34 [594112,63931] 0 2024-03-11T19:56:18.572 INFO:tasks.workunit.client.0.smithi107.stdout:9/646: readlink d4/d9/d9f/ld5 0 2024-03-11T19:56:18.572 INFO:tasks.workunit.client.0.smithi107.stdout:9/647: dread - d4/fbf zero size 2024-03-11T19:56:18.572 INFO:tasks.workunit.client.0.smithi107.stdout:9/648: fsync d4/fbf 0 2024-03-11T19:56:18.575 INFO:tasks.workunit.client.0.smithi107.stdout:0/658: dread de/d11/d23/d32/d34/f6c [0,4194304] 0 2024-03-11T19:56:18.575 INFO:tasks.workunit.client.0.smithi107.stdout:0/659: dread - de/d11/d23/d24/d4f/fc7 zero size 2024-03-11T19:56:18.575 INFO:tasks.workunit.client.0.smithi107.stdout:3/732: link d1/d10/d4e/d73/c40 d1/d10/d4e/ce5 0 2024-03-11T19:56:18.586 INFO:tasks.workunit.client.0.smithi107.stdout:3/733: symlink d1/d10/d13/d4a/le6 0 2024-03-11T19:56:18.587 INFO:tasks.workunit.client.0.smithi107.stdout:3/734: symlink d1/d10/d21/d43/d48/d98/le7 0 2024-03-11T19:56:18.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/649: dread d4/d1d/d1f/f39 [0,4194304] 0 2024-03-11T19:56:18.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/650: chown d4/d9/d9f/da7/ddb 88 1 2024-03-11T19:56:18.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/651: dread - d4/d1c/d47/f9c zero size 2024-03-11T19:56:18.603 INFO:tasks.workunit.client.0.smithi107.stdout:9/652: mknod d4/d1c/d47/ce6 0 2024-03-11T19:56:18.604 INFO:tasks.workunit.client.0.smithi107.stdout:9/653: link d4/d1d/d1f/d28/f91 d4/d1d/d1f/d28/fe7 0 2024-03-11T19:56:18.614 INFO:tasks.workunit.client.0.smithi107.stdout:3/735: dread d1/d10/d13/f2c [0,4194304] 0 2024-03-11T19:56:18.615 INFO:tasks.workunit.client.0.smithi107.stdout:3/736: fsync d1/d10/d26/d17/fe2 0 2024-03-11T19:56:18.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/660: dwrite fd [0,4194304] 0 2024-03-11T19:56:18.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/737: mknod d1/dbb/de3/ce8 0 2024-03-11T19:56:18.620 INFO:tasks.workunit.client.0.smithi107.stdout:3/738: creat d1/d10/d21/d43/d48/fe9 x:0 0 0 2024-03-11T19:56:18.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/739: symlink d1/d10/d13/d4a/d4c/d49/lea 0 2024-03-11T19:56:18.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/740: truncate d1/d10/d13/d6c/d2d/d4d/f71 1551044 0 2024-03-11T19:56:18.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/604: dwrite d10/d16/d2b/da5/faa [0,4194304] 0 2024-03-11T19:56:18.629 INFO:tasks.workunit.client.0.smithi107.stdout:3/741: rmdir d1/d10/d13/d6c/d2d/d4d/d91/da5 39 2024-03-11T19:56:18.631 INFO:tasks.workunit.client.0.smithi107.stdout:3/742: symlink d1/d10/d13/d6c/d2d/d4d/d91/db4/leb 0 2024-03-11T19:56:18.632 INFO:tasks.workunit.client.0.smithi107.stdout:3/743: mknod d1/d10/d21/d43/d48/cec 0 2024-03-11T19:56:18.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/744: truncate d1/d10/d13/d6c/d2d/d4d/d91/db4/fd8 523474 0 2024-03-11T19:56:18.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/745: creat d1/d10/d13/d6c/d2d/d4d/d91/db4/fed x:0 0 0 2024-03-11T19:56:18.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/746: mknod d1/d10/d4e/cee 0 2024-03-11T19:56:18.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/747: symlink d1/d10/d21/d43/d48/d98/lef 0 2024-03-11T19:56:18.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/748: unlink d1/d10/l5f 0 2024-03-11T19:56:18.640 INFO:tasks.workunit.client.0.smithi107.stdout:3/749: mkdir d1/d10/d13/d6c/d2d/d4d/d91/df0 0 2024-03-11T19:56:18.640 INFO:tasks.workunit.client.0.smithi107.stdout:3/750: fsync d1/d10/d26/d17/f81 0 2024-03-11T19:56:18.642 INFO:tasks.workunit.client.0.smithi107.stdout:3/751: mknod d1/dbb/cf1 0 2024-03-11T19:56:18.643 INFO:tasks.workunit.client.0.smithi107.stdout:5/736: rename da/d14/f5e to da/d14/d38/d59/d5f/dc9/fe1 0 2024-03-11T19:56:18.644 INFO:tasks.workunit.client.0.smithi107.stdout:5/737: write da/d31/d42/f11 [12426369,17731] 0 2024-03-11T19:56:18.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/752: unlink d1/d10/d13/d6c/d2d/l8f 0 2024-03-11T19:56:18.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/738: unlink da/d14/d18/c9d 0 2024-03-11T19:56:18.670 INFO:tasks.workunit.client.0.smithi107.stdout:0/661: dwrite de/d11/d23/f9d [0,4194304] 0 2024-03-11T19:56:18.676 INFO:tasks.workunit.client.0.smithi107.stdout:6/605: rename d10/d19/d38/d60/db8/cc0 to d10/d16/d1f/d51/d76/d7f/ccb 0 2024-03-11T19:56:18.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/753: write d1/d10/d13/d4a/d4c/f7b [1778305,65047] 0 2024-03-11T19:56:18.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/739: dwrite da/fcc [0,4194304] 0 2024-03-11T19:56:18.688 INFO:tasks.workunit.client.0.smithi107.stdout:5/740: read da/d14/d18/d1e/d9b/fc0 [462918,35174] 0 2024-03-11T19:56:18.688 INFO:tasks.workunit.client.0.smithi107.stdout:5/741: fsync da/d14/d18/d1e/d9b/d73/faa 0 2024-03-11T19:56:18.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/742: creat da/d14/d18/d1e/d9b/d72/dc8/fe2 x:0 0 0 2024-03-11T19:56:18.691 INFO:tasks.workunit.client.0.smithi107.stdout:5/743: fdatasync da/d14/d38/d59/d5f/dc9/fb8 0 2024-03-11T19:56:18.692 INFO:tasks.workunit.client.0.smithi107.stdout:5/744: symlink da/d14/d38/le3 0 2024-03-11T19:56:18.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/745: chown da/d14/d18/d44/dcd 3 1 2024-03-11T19:56:18.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/746: dread - da/d14/d18/f95 zero size 2024-03-11T19:56:18.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/747: truncate da/d14/d18/d6d/fa7 780429 0 2024-03-11T19:56:18.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/748: mknod da/d14/d18/d44/dcd/ce4 0 2024-03-11T19:56:18.695 INFO:tasks.workunit.client.0.smithi107.stdout:5/749: truncate da/d14/d18/f2c 1146442 0 2024-03-11T19:56:18.695 INFO:tasks.workunit.client.0.smithi107.stdout:5/750: fsync da/d14/d18/d1e/d9b/fa4 0 2024-03-11T19:56:18.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/751: mkdir da/d14/d38/d59/de5 0 2024-03-11T19:56:18.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/752: mknod da/d14/ce6 0 2024-03-11T19:56:18.710 INFO:tasks.workunit.client.0.smithi107.stdout:0/662: dwrite de/d11/d23/d32/d34/f82 [0,4194304] 0 2024-03-11T19:56:18.730 INFO:tasks.workunit.client.0.smithi107.stdout:0/663: dread de/f2f [0,4194304] 0 2024-03-11T19:56:18.731 INFO:tasks.workunit.client.0.smithi107.stdout:0/664: readlink l2 0 2024-03-11T19:56:18.743 INFO:tasks.workunit.client.0.smithi107.stdout:5/753: dwrite da/d14/d43/fce [0,4194304] 0 2024-03-11T19:56:19.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:18 smithi122 ceph-mon[36397]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 257 op/s 2024-03-11T19:56:19.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:18 smithi145 ceph-mon[36730]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 257 op/s 2024-03-11T19:56:19.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:18 smithi107 ceph-mon[31712]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 257 op/s 2024-03-11T19:56:19.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/606: rename d10/d16/d24/f58 to d10/d19/d38/d60/db8/fcc 0 2024-03-11T19:56:19.551 INFO:tasks.workunit.client.0.smithi107.stdout:6/607: getdents d10/d16/d24/d2e/d9d 0 2024-03-11T19:56:19.552 INFO:tasks.workunit.client.0.smithi107.stdout:3/754: rename d1/d10/d21/d43/l7a to d1/dbb/de3/lf2 0 2024-03-11T19:56:19.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/608: creat d10/d16/fcd x:0 0 0 2024-03-11T19:56:19.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/609: getdents d10/d16/d2b/da5 0 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:5/754: sync 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:0/665: sync 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:4/657: sync 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/701: sync 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:5/755: stat da/d14/d18/d1e/d9b/db3 0 2024-03-11T19:56:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/610: sync 2024-03-11T19:56:19.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/744: sync 2024-03-11T19:56:19.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/654: sync 2024-03-11T19:56:19.586 INFO:tasks.workunit.client.0.smithi107.stdout:8/548: sync 2024-03-11T19:56:19.587 INFO:tasks.workunit.client.0.smithi107.stdout:5/756: chown da/d14/d18/d1e/fb6 0 1 2024-03-11T19:56:19.591 INFO:tasks.workunit.client.0.smithi107.stdout:2/611: symlink d1/d21/lcc 0 2024-03-11T19:56:19.591 INFO:tasks.workunit.client.0.smithi107.stdout:0/666: truncate de/d11/d23/d24/f69 3555469 0 2024-03-11T19:56:19.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/667: write de/d11/d23/d24/f60 [1719091,63454] 0 2024-03-11T19:56:19.592 INFO:tasks.workunit.client.0.smithi107.stdout:7/745: rename d6/da/d16/d91/d97/d9b/cac to d6/da/d16/d5c/d7d/d60/d61/d9c/cea 0 2024-03-11T19:56:19.592 INFO:tasks.workunit.client.0.smithi107.stdout:7/746: write d6/da/f1f [3553497,44111] 0 2024-03-11T19:56:19.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/655: truncate d4/d9/dae/f77 3362989 0 2024-03-11T19:56:19.595 INFO:tasks.workunit.client.0.smithi107.stdout:2/612: mknod d1/d1a/d1e/d55/ccd 0 2024-03-11T19:56:19.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/668: mkdir de/d9e/dc8 0 2024-03-11T19:56:19.597 INFO:tasks.workunit.client.0.smithi107.stdout:7/747: link d6/da/d16/d5c/d7d/l5d d6/da/leb 0 2024-03-11T19:56:19.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/613: rmdir d1/d1a/d34/d4f/d63 39 2024-03-11T19:56:19.599 INFO:tasks.workunit.client.0.smithi107.stdout:3/755: dread d1/f9d [0,4194304] 0 2024-03-11T19:56:19.603 INFO:tasks.workunit.client.0.smithi107.stdout:5/757: rmdir da/d14/d18/d44/da5 39 2024-03-11T19:56:19.608 INFO:tasks.workunit.client.0.smithi107.stdout:9/656: mknod d4/d1d/d1f/d28/d2c/ce8 0 2024-03-11T19:56:19.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/614: symlink d1/d1a/d34/d4f/d63/db6/lce 0 2024-03-11T19:56:19.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/756: rename d1/d10/d13/d4a/d4c/f7b to d1/d10/d13/d4a/d4c/ff3 0 2024-03-11T19:56:19.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/615: creat d1/d1a/d1e/d41/fcf x:0 0 0 2024-03-11T19:56:19.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/758: creat da/fe7 x:0 0 0 2024-03-11T19:56:19.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/759: unlink da/d14/d18/d1e/fb6 0 2024-03-11T19:56:19.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/610: write fc [2885981,63019] 0 2024-03-11T19:56:19.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/611: fsync d10/d3d/d54/f80 0 2024-03-11T19:56:19.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/612: creat d10/d19/d38/fce x:0 0 0 2024-03-11T19:56:19.614 INFO:tasks.workunit.client.0.smithi107.stdout:5/760: rename da/d6e/c7b to da/d14/d38/d46/dd9/ce8 0 2024-03-11T19:56:19.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/761: getdents da/d14/d43/d9e/dad 0 2024-03-11T19:56:19.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/762: creat da/d14/d38/fe9 x:0 0 0 2024-03-11T19:56:19.624 INFO:tasks.workunit.client.0.smithi107.stdout:6/613: creat d10/d16/d2b/d39/fcf x:0 0 0 2024-03-11T19:56:19.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/549: dread dd/de/d24/d28/f4a [0,4194304] 0 2024-03-11T19:56:19.626 INFO:tasks.workunit.client.0.smithi107.stdout:9/657: dread d4/d1c/d37/f50 [0,4194304] 0 2024-03-11T19:56:19.627 INFO:tasks.workunit.client.0.smithi107.stdout:9/658: readlink d4/d1d/d1f/l36 0 2024-03-11T19:56:19.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/550: rmdir dd/de/d24/d86 0 2024-03-11T19:56:19.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/551: chown dd/d13/d32/f89 21048540 1 2024-03-11T19:56:19.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/552: write dd/d13/d18/d81/f8d [296192,50231] 0 2024-03-11T19:56:19.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/553: creat dd/de/d1b/f9a x:0 0 0 2024-03-11T19:56:19.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/554: chown dd/de/d1b/d2c/l38 402909644 1 2024-03-11T19:56:19.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/555: fsync dd/d13/d18/f75 0 2024-03-11T19:56:19.631 INFO:tasks.workunit.client.0.smithi107.stdout:3/757: dread d1/d10/d21/d43/d48/f50 [0,4194304] 0 2024-03-11T19:56:19.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/763: mknod da/d14/d43/d9e/dad/cea 0 2024-03-11T19:56:19.632 INFO:tasks.workunit.client.0.smithi107.stdout:8/556: symlink dd/de/d1b/d2c/l9b 0 2024-03-11T19:56:19.635 INFO:tasks.workunit.client.0.smithi107.stdout:4/658: dread d4/d7/f17 [0,4194304] 0 2024-03-11T19:56:19.638 INFO:tasks.workunit.client.0.smithi107.stdout:4/659: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/cd4 0 2024-03-11T19:56:19.638 INFO:tasks.workunit.client.0.smithi107.stdout:4/660: write d4/d7/db/d5c/fc4 [197265,110048] 0 2024-03-11T19:56:19.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/557: mknod dd/de/d49/d7e/d92/c9c 0 2024-03-11T19:56:19.639 INFO:tasks.workunit.client.0.smithi107.stdout:4/661: fsync d4/fc9 0 2024-03-11T19:56:19.639 INFO:tasks.workunit.client.0.smithi107.stdout:4/662: dread - d4/d7/d1f/d2b/d19/d75/d8e/fbc zero size 2024-03-11T19:56:19.640 INFO:tasks.workunit.client.0.smithi107.stdout:8/558: creat dd/d13/d18/f9d x:0 0 0 2024-03-11T19:56:19.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/748: dread d6/da/d16/f3f [0,4194304] 0 2024-03-11T19:56:19.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/749: chown d6/da/d16/d5c/d7d/d60/d61/d9c/cea 15588 1 2024-03-11T19:56:19.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/663: link d4/d7/d1f/d2b/d24/d26/d29/d36/f57 d4/d7/d1f/d2b/d24/fd5 0 2024-03-11T19:56:19.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/664: mknod d4/d7/d1f/d2b/d24/d26/d29/cd6 0 2024-03-11T19:56:19.645 INFO:tasks.workunit.client.0.smithi107.stdout:4/665: symlink d4/d7/d2f/d33/ld7 0 2024-03-11T19:56:19.646 INFO:tasks.workunit.client.0.smithi107.stdout:3/758: dread d1/d10/d26/f58 [0,4194304] 0 2024-03-11T19:56:19.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/666: link d4/d7/d2f/d3b/f58 d4/d7/d1f/d2b/d19/d9a/fd8 0 2024-03-11T19:56:19.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/759: rename d1/d10/d4e/d73/cbd to d1/d10/d13/cf4 0 2024-03-11T19:56:19.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/760: dread - d1/d10/d21/d43/d4f/f92 zero size 2024-03-11T19:56:19.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/761: chown d1/d10/d21/d43/d48/cec 7117992 1 2024-03-11T19:56:19.653 INFO:tasks.workunit.client.0.smithi107.stdout:3/762: creat d1/d10/d21/ff5 x:0 0 0 2024-03-11T19:56:19.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/763: dread - d1/d10/d26/d17/fe2 zero size 2024-03-11T19:56:19.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/764: truncate d1/f5 8902393 0 2024-03-11T19:56:19.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/765: chown d1/d10/d13/d6c/d2d/d4d/cb2 164234712 1 2024-03-11T19:56:19.668 INFO:tasks.workunit.client.0.smithi107.stdout:9/659: read d4/d9/d9f/fbc [2922509,19730] 0 2024-03-11T19:56:19.668 INFO:tasks.workunit.client.0.smithi107.stdout:9/660: stat d4/f58 0 2024-03-11T19:56:19.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/661: mkdir d4/d1c/d37/dd9/de9 0 2024-03-11T19:56:19.670 INFO:tasks.workunit.client.0.smithi107.stdout:0/669: dwrite de/d7a/fb7 [0,4194304] 0 2024-03-11T19:56:19.670 INFO:tasks.workunit.client.0.smithi107.stdout:0/670: dread - de/d11/d23/d32/d34/d39/f6e zero size 2024-03-11T19:56:19.673 INFO:tasks.workunit.client.0.smithi107.stdout:2/616: dwrite d1/d1a/d34/d4f/f5d [4194304,4194304] 0 2024-03-11T19:56:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:5/764: link da/l99 da/d14/d18/d1e/d9b/leb 0 2024-03-11T19:56:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/614: read d10/d16/f20 [171545,27686] 0 2024-03-11T19:56:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:5/765: link da/d14/d43/d47/f91 da/d14/d43/d7f/fec 0 2024-03-11T19:56:19.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/766: rename da/d14/d18/d1e/d9b/fc0 to da/d14/d18/d1e/d9b/d72/fed 0 2024-03-11T19:56:19.707 INFO:tasks.workunit.client.0.smithi107.stdout:5/767: rename da/d14/d38/d59/f1d to da/d14/d43/dab/fee 0 2024-03-11T19:56:19.723 INFO:tasks.workunit.client.0.smithi107.stdout:4/667: dwrite d4/d7/d1f/d2b/d19/f23 [4194304,4194304] 0 2024-03-11T19:56:19.723 INFO:tasks.workunit.client.0.smithi107.stdout:4/668: link d4/d7/d1f/d2b/c86 d4/d7/da0/dcb/cd9 0 2024-03-11T19:56:19.723 INFO:tasks.workunit.client.0.smithi107.stdout:4/669: readlink d4/d7/d1f/l40 0 2024-03-11T19:56:19.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/670: stat d4/d7/db/fc2 0 2024-03-11T19:56:19.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/671: write d4/d7/d1f/d2b/d24/d26/f43 [8593755,35458] 0 2024-03-11T19:56:19.725 INFO:tasks.workunit.client.0.smithi107.stdout:4/672: creat d4/d7/db/d5c/fda x:0 0 0 2024-03-11T19:56:19.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/673: symlink d4/d7/d1f/d2b/ldb 0 2024-03-11T19:56:19.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/766: dwrite d1/d10/d2a/f38 [0,4194304] 0 2024-03-11T19:56:19.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/767: link d1/d10/d26/l89 d1/d10/d4e/lf6 0 2024-03-11T19:56:19.738 INFO:tasks.workunit.client.0.smithi107.stdout:9/662: dwrite d4/f58 [0,4194304] 0 2024-03-11T19:56:19.742 INFO:tasks.workunit.client.0.smithi107.stdout:0/671: dwrite de/d11/d23/d32/d34/d39/f53 [0,4194304] 0 2024-03-11T19:56:19.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/663: mknod d4/d1c/d37/dd9/de9/cea 0 2024-03-11T19:56:19.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/664: readlink d4/d1d/d1f/d28/l55 0 2024-03-11T19:56:19.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/750: dwrite d6/da/d16/f6f [0,4194304] 0 2024-03-11T19:56:19.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/559: dread dd/d13/d32/f7b [0,4194304] 0 2024-03-11T19:56:19.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/751: dread - d6/d9/d57/f9e zero size 2024-03-11T19:56:19.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/752: readlink d6/d9/d57/l9f 0 2024-03-11T19:56:19.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/768: dread d1/d10/d4e/d73/d8d/fc1 [0,4194304] 0 2024-03-11T19:56:19.746 INFO:tasks.workunit.client.0.smithi107.stdout:8/560: creat dd/de/d1b/d20/f9e x:0 0 0 2024-03-11T19:56:19.747 INFO:tasks.workunit.client.0.smithi107.stdout:6/615: link d10/d16/d24/d2e/f86 d10/d16/d1f/d51/d76/d7f/fd0 0 2024-03-11T19:56:19.748 INFO:tasks.workunit.client.0.smithi107.stdout:0/672: dread de/f67 [0,4194304] 0 2024-03-11T19:56:19.748 INFO:tasks.workunit.client.0.smithi107.stdout:6/616: creat d10/d16/d2b/d39/d6a/fd1 x:0 0 0 2024-03-11T19:56:19.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/617: chown d10/d16/d1f/d51/d7a 0 1 2024-03-11T19:56:19.750 INFO:tasks.workunit.client.0.smithi107.stdout:6/618: write d10/d16/d2b/d39/d6a/f83 [5190668,6601] 0 2024-03-11T19:56:19.750 INFO:tasks.workunit.client.0.smithi107.stdout:9/665: mknod d4/d9/ceb 0 2024-03-11T19:56:19.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/753: symlink d6/da/d16/d5c/d7d/d60/dd2/lec 0 2024-03-11T19:56:19.762 INFO:tasks.workunit.client.0.smithi107.stdout:4/674: rename d4/d7/d1f/d2b/f5d to d4/d7/d2f/d3c/fdc 0 2024-03-11T19:56:19.765 INFO:tasks.workunit.client.0.smithi107.stdout:3/769: rename d1/d10/d13/d4a/f93 to d1/d10/d13/d6c/d5d/d76/ff7 0 2024-03-11T19:56:19.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/770: fdatasync d1/d10/d13/d6c/f44 0 2024-03-11T19:56:19.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/771: fsync d1/d10/f57 0 2024-03-11T19:56:19.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/772: stat d1/d10/d21/d43/d4f 0 2024-03-11T19:56:19.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/675: creat d4/d7/d1f/d2b/d19/d75/d8e/fdd x:0 0 0 2024-03-11T19:56:19.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/676: chown d4/d7/da0 11636 1 2024-03-11T19:56:19.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/666: rename d4/d1d/d1f/d28/l55 to d4/d1c/lec 0 2024-03-11T19:56:19.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/677: unlink d4/d7/d1f/d2b/d24/d26/fae 0 2024-03-11T19:56:19.774 INFO:tasks.workunit.client.0.smithi107.stdout:8/561: dread dd/d13/d18/f37 [0,4194304] 0 2024-03-11T19:56:19.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/678: dread - d4/d7/d1f/d2b/d93/dc3/dd0/fbf zero size 2024-03-11T19:56:19.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/679: dread - d4/d7/d1f/d2b/d19/fa2 zero size 2024-03-11T19:56:19.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/768: dwrite da/d14/d38/d59/dbd/fde [0,4194304] 0 2024-03-11T19:56:19.792 INFO:tasks.workunit.client.0.smithi107.stdout:0/673: dwrite de/d7a/fb7 [0,4194304] 0 2024-03-11T19:56:19.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/674: write de/f67 [40635,111760] 0 2024-03-11T19:56:19.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/675: chown de/d11/d23/d26/l87 1008956164 1 2024-03-11T19:56:19.809 INFO:tasks.workunit.client.0.smithi107.stdout:3/773: read d1/d10/d21/d43/d48/f56 [217247,56173] 0 2024-03-11T19:56:19.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/774: unlink d1/d10/d13/d6c/d5d/d76/lc5 0 2024-03-11T19:56:19.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/775: creat d1/d10/d2a/d7e/da0/ff8 x:0 0 0 2024-03-11T19:56:19.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/667: dread d4/d1d/d1f/d28/d2c/f49 [0,4194304] 0 2024-03-11T19:56:19.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/668: dread - d4/d9/dae/fe3 zero size 2024-03-11T19:56:19.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/669: dread - d4/d1d/d5c/d64/db3/fc8 zero size 2024-03-11T19:56:19.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/670: fdatasync d4/d1d/d1f/d28/f6e 0 2024-03-11T19:56:19.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/671: mknod d4/d1d/d5c/d64/ced 0 2024-03-11T19:56:19.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/562: dwrite dd/d13/f34 [0,4194304] 0 2024-03-11T19:56:19.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/563: truncate dd/d13/f34 1411745 0 2024-03-11T19:56:19.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/564: fsync dd/de/d49/f2b 0 2024-03-11T19:56:19.852 INFO:tasks.workunit.client.0.smithi107.stdout:8/565: symlink dd/de/d1b/d20/l9f 0 2024-03-11T19:56:19.853 INFO:tasks.workunit.client.0.smithi107.stdout:0/676: symlink de/d11/d74/lc9 0 2024-03-11T19:56:19.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/619: fsync d10/d16/d2b/d39/d6a/fd1 0 2024-03-11T19:56:19.855 INFO:tasks.workunit.client.0.smithi107.stdout:8/566: symlink dd/de/d24/d28/d47/la0 0 2024-03-11T19:56:19.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/567: readlink dd/d13/d32/d51/l5d 0 2024-03-11T19:56:19.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/568: fsync dd/de/d1b/d20/f29 0 2024-03-11T19:56:19.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/677: rmdir de/d7a 39 2024-03-11T19:56:19.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/678: mkdir de/d11/d23/d26/dca 0 2024-03-11T19:56:19.858 INFO:tasks.workunit.client.0.smithi107.stdout:4/680: dwrite d4/d7/db/d5c/fc4 [0,4194304] 0 2024-03-11T19:56:19.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/776: dwrite d1/d10/f6f [0,4194304] 0 2024-03-11T19:56:19.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/777: dread - d1/d10/d4e/fd3 zero size 2024-03-11T19:56:19.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/620: unlink d10/d16/d24/d2e/f46 0 2024-03-11T19:56:19.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/621: chown d10/d16/fcd 1380 1 2024-03-11T19:56:19.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/769: link da/d14/d18/d1e/d9b/d73/fa3 da/d14/d38/fef 0 2024-03-11T19:56:19.876 INFO:tasks.workunit.client.0.smithi107.stdout:3/778: dread d1/d45/f86 [4194304,4194304] 0 2024-03-11T19:56:19.877 INFO:tasks.workunit.client.0.smithi107.stdout:3/779: mkdir d1/d10/d13/d4a/d4c/d49/dca/df9 0 2024-03-11T19:56:19.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/780: unlink d1/d10/d4e/cee 0 2024-03-11T19:56:19.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/781: getdents d1/d10/d4e/d8c 0 2024-03-11T19:56:19.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/782: stat d1/d10/d13/d6c/d5d/l77 0 2024-03-11T19:56:19.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/783: dread - d1/d10/d4e/d73/fc3 zero size 2024-03-11T19:56:19.880 INFO:tasks.workunit.client.0.smithi107.stdout:3/784: rmdir d1/d10/d4e/d73 39 2024-03-11T19:56:19.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/702: sync 2024-03-11T19:56:19.884 INFO:tasks.workunit.client.0.smithi107.stdout:1/703: mknod d6/d28/dc4/ce3 0 2024-03-11T19:56:19.884 INFO:tasks.workunit.client.0.smithi107.stdout:1/704: readlink d6/d28/dc4/lb8 0 2024-03-11T19:56:19.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/672: dwrite d4/d1d/d1f/d28/d5d/f7f [0,4194304] 0 2024-03-11T19:56:19.892 INFO:tasks.workunit.client.0.smithi107.stdout:1/705: link d6/d7e/d81/d27/d2c/l42 d6/d7e/d81/d27/d2c/d7d/le4 0 2024-03-11T19:56:19.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/785: read d1/d10/d13/d6c/d2d/d4d/d91/db4/fb8 [668088,26099] 0 2024-03-11T19:56:19.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/786: fdatasync d1/d10/d2a/d7e/da0/ff8 0 2024-03-11T19:56:19.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/787: creat d1/dbb/ffa x:0 0 0 2024-03-11T19:56:19.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/681: dwrite d4/d7/d1f/d2b/d24/d26/d29/d36/f7a [0,4194304] 0 2024-03-11T19:56:19.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/788: stat d1/d10/d4e/d73/c40 0 2024-03-11T19:56:19.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/789: write d1/d10/d21/d43/d48/f50 [2563024,123030] 0 2024-03-11T19:56:19.914 INFO:tasks.workunit.client.0.smithi107.stdout:3/790: unlink d1/d10/d13/d4a/d4c/c7d 0 2024-03-11T19:56:19.920 INFO:tasks.workunit.client.0.smithi107.stdout:3/791: creat d1/d10/d13/d6c/d2d/d4d/ffb x:0 0 0 2024-03-11T19:56:19.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/770: dwrite da/d14/d38/d59/d5f/f6a [0,4194304] 0 2024-03-11T19:56:19.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/771: creat da/d14/d18/ff0 x:0 0 0 2024-03-11T19:56:19.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/673: write d4/d1d/d1f/d28/d2c/fa5 [3915786,35551] 0 2024-03-11T19:56:19.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/679: dwrite de/d11/f2c [4194304,4194304] 0 2024-03-11T19:56:19.932 INFO:tasks.workunit.client.0.smithi107.stdout:6/622: dread d10/d19/d38/f5c [0,4194304] 0 2024-03-11T19:56:19.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/623: write d10/d19/d3b/f73 [19825,114151] 0 2024-03-11T19:56:19.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/680: symlink de/d11/d23/d24/da4/lcb 0 2024-03-11T19:56:19.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/681: mknod de/d11/d23/d24/da4/ccc 0 2024-03-11T19:56:19.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/682: unlink de/f2f 0 2024-03-11T19:56:19.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/683: chown de/c15 0 1 2024-03-11T19:56:19.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/684: stat f5 0 2024-03-11T19:56:19.950 INFO:tasks.workunit.client.0.smithi107.stdout:9/674: write d4/d9/dae/d6d/db0/fc2 [1956272,93835] 0 2024-03-11T19:56:19.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/569: rename dd/de/d49/f44 to dd/de/d1b/d20/d5e/fa1 0 2024-03-11T19:56:19.953 INFO:tasks.workunit.client.0.smithi107.stdout:2/617: sync 2024-03-11T19:56:19.953 INFO:tasks.workunit.client.0.smithi107.stdout:2/618: chown d1/d1a/d1e/d26/l52 22535 1 2024-03-11T19:56:19.953 INFO:tasks.workunit.client.0.smithi107.stdout:2/619: chown d1/d1a/d1e/l90 702 1 2024-03-11T19:56:19.954 INFO:tasks.workunit.client.0.smithi107.stdout:8/570: symlink dd/de/d1b/d2c/la2 0 2024-03-11T19:56:19.955 INFO:tasks.workunit.client.0.smithi107.stdout:8/571: creat dd/de/d1b/d20/d56/fa3 x:0 0 0 2024-03-11T19:56:19.955 INFO:tasks.workunit.client.0.smithi107.stdout:8/572: dread - dd/de/d1b/d20/f9e zero size 2024-03-11T19:56:19.956 INFO:tasks.workunit.client.0.smithi107.stdout:2/620: creat d1/d21/dbb/fd0 x:0 0 0 2024-03-11T19:56:19.956 INFO:tasks.workunit.client.0.smithi107.stdout:2/621: fsync d1/d1a/d34/d4f/f5d 0 2024-03-11T19:56:19.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/573: mkdir dd/de/d1b/d20/d56/da4 0 2024-03-11T19:56:19.959 INFO:tasks.workunit.client.0.smithi107.stdout:9/675: write d4/d1c/d37/f50 [959411,89456] 0 2024-03-11T19:56:19.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/676: chown d4/d1d/d1f/d28/l29 543 1 2024-03-11T19:56:19.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/706: dwrite d6/d7e/d81/d27/f3e [0,4194304] 0 2024-03-11T19:56:19.971 INFO:tasks.workunit.client.0.smithi107.stdout:6/624: write d10/d16/d24/d2e/f2f [1545780,63494] 0 2024-03-11T19:56:19.971 INFO:tasks.workunit.client.0.smithi107.stdout:6/625: write d10/d16/d2b/f59 [556015,123206] 0 2024-03-11T19:56:19.971 INFO:tasks.workunit.client.0.smithi107.stdout:6/626: stat d10/d19/f31 0 2024-03-11T19:56:19.978 INFO:tasks.workunit.client.0.smithi107.stdout:6/627: rename d10/d3d/d54/f96 to d10/d3d/d54/fd2 0 2024-03-11T19:56:19.980 INFO:tasks.workunit.client.0.smithi107.stdout:5/772: dwrite da/d14/d18/d1e/d9b/d73/f8a [0,4194304] 0 2024-03-11T19:56:19.980 INFO:tasks.workunit.client.0.smithi107.stdout:5/773: chown c1 220119 1 2024-03-11T19:56:19.983 INFO:tasks.workunit.client.0.smithi107.stdout:0/685: rename de/d11/d23/d24/dc6 to de/d11/d23/d24/da4/dae/dcd 0 2024-03-11T19:56:19.985 INFO:tasks.workunit.client.0.smithi107.stdout:6/628: mknod d10/d16/d1f/cd3 0 2024-03-11T19:56:19.985 INFO:tasks.workunit.client.0.smithi107.stdout:6/629: truncate d10/d16/d1f/d51/d7a/dba/fc7 744503 0 2024-03-11T19:56:19.986 INFO:tasks.workunit.client.0.smithi107.stdout:6/630: truncate d10/d16/d1f/d51/f87 534547 0 2024-03-11T19:56:19.987 INFO:tasks.workunit.client.0.smithi107.stdout:6/631: getdents d10/d19/d3b/d70/d8a 0 2024-03-11T19:56:19.987 INFO:tasks.workunit.client.0.smithi107.stdout:1/707: rename d6/d28/f3f to d6/d7e/d81/fe5 0 2024-03-11T19:56:19.990 INFO:tasks.workunit.client.0.smithi107.stdout:4/682: dwrite d4/d7/d1f/d2b/d19/f23 [0,4194304] 0 2024-03-11T19:56:19.990 INFO:tasks.workunit.client.0.smithi107.stdout:1/708: rename d6/d7e/da4/laf to d6/d7e/da5/le6 0 2024-03-11T19:56:19.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/709: mkdir d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7 0 2024-03-11T19:56:19.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/710: chown d6/d7e/d81/d27/d2c/d32/l70 978 1 2024-03-11T19:56:19.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/711: write d6/d19/f73 [3744228,15539] 0 2024-03-11T19:56:19.993 INFO:tasks.workunit.client.0.smithi107.stdout:2/622: write d1/d38/d75/f89 [46411,79739] 0 2024-03-11T19:56:19.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/574: dread dd/de/d1b/d20/d5e/f80 [0,4194304] 0 2024-03-11T19:56:19.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/712: unlink d6/d7e/d81/d27/d2c/d32/f6f 0 2024-03-11T19:56:19.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/575: creat dd/de/d49/d46/fa5 x:0 0 0 2024-03-11T19:56:19.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/623: mkdir d1/d1d/d64/d97/dd1 0 2024-03-11T19:56:19.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/624: chown d1/d1a/d34/d4f/l93 125651499 1 2024-03-11T19:56:20.001 INFO:tasks.workunit.client.0.smithi107.stdout:2/625: mknod d1/d1a/d34/d4f/cd2 0 2024-03-11T19:56:20.001 INFO:tasks.workunit.client.0.smithi107.stdout:2/626: chown d1/d1d/d64/f99 1840740 1 2024-03-11T19:56:20.008 INFO:tasks.workunit.client.0.smithi107.stdout:3/792: dwrite d1/d10/d13/d6c/d5d/d76/fc8 [0,4194304] 0 2024-03-11T19:56:20.008 INFO:tasks.workunit.client.0.smithi107.stdout:3/793: truncate d1/d10/d13/d4a/d4c/d49/dca/fce 529305 0 2024-03-11T19:56:20.009 INFO:tasks.workunit.client.0.smithi107.stdout:7/754: sync 2024-03-11T19:56:20.011 INFO:tasks.workunit.client.0.smithi107.stdout:2/627: write d1/d1a/d34/d4f/d63/f66 [4854397,435] 0 2024-03-11T19:56:20.015 INFO:tasks.workunit.client.0.smithi107.stdout:2/628: rename d1/d1d/d64/f8a to d1/d1d/d80/fd3 0 2024-03-11T19:56:20.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/576: read dd/d16/f7f [3756960,114958] 0 2024-03-11T19:56:20.025 INFO:tasks.workunit.client.0.smithi107.stdout:7/755: dread d6/da/d16/d5c/d7d/f5e [0,4194304] 0 2024-03-11T19:56:20.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/577: mknod dd/de/d49/d46/ca6 0 2024-03-11T19:56:20.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/578: fdatasync dd/de/d49/f5b 0 2024-03-11T19:56:20.025 INFO:tasks.workunit.client.0.smithi107.stdout:3/794: dread d1/d10/d13/d6c/d5d/d76/ff7 [0,4194304] 0 2024-03-11T19:56:20.026 INFO:tasks.workunit.client.0.smithi107.stdout:3/795: creat d1/d10/d21/d43/d48/ffc x:0 0 0 2024-03-11T19:56:20.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/713: read d6/f54 [2473340,67531] 0 2024-03-11T19:56:20.041 INFO:tasks.workunit.client.0.smithi107.stdout:2/629: dread d1/d21/f28 [0,4194304] 0 2024-03-11T19:56:20.041 INFO:tasks.workunit.client.0.smithi107.stdout:2/630: chown d1/d1a/d34/d4f/d63/l8b 1053691043 1 2024-03-11T19:56:20.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/774: write da/d14/d43/d47/fc7 [2475690,87583] 0 2024-03-11T19:56:20.046 INFO:tasks.workunit.client.0.smithi107.stdout:2/631: symlink d1/d1a/d1e/d55/ld4 0 2024-03-11T19:56:20.046 INFO:tasks.workunit.client.0.smithi107.stdout:2/632: truncate d1/d38/d75/fb0 4378621 0 2024-03-11T19:56:20.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/775: creat da/d31/ff1 x:0 0 0 2024-03-11T19:56:20.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/776: creat da/d14/d18/d1e/d9b/d73/ff2 x:0 0 0 2024-03-11T19:56:20.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/777: chown da/d14/d18/d1e/d9b/leb 58 1 2024-03-11T19:56:20.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/796: dread d1/d10/d21/d43/d48/f8e [0,4194304] 0 2024-03-11T19:56:20.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/633: creat d1/d1a/d34/d4f/fd5 x:0 0 0 2024-03-11T19:56:20.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/797: rename d1/d10/d13/d6c/d2d/d4d/d91/db4 to d1/dbb/de3/dfd 0 2024-03-11T19:56:20.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/634: rmdir d1/d1a/d34/d4f/db3 39 2024-03-11T19:56:20.064 INFO:tasks.workunit.client.0.smithi107.stdout:3/798: getdents d1/dbb 0 2024-03-11T19:56:20.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/799: chown d1/d10/d13/d6c/d2d/d4d/d91/da5 67 1 2024-03-11T19:56:20.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/800: fsync d1/d10/d13/d6c/d5d/f94 0 2024-03-11T19:56:20.069 INFO:tasks.workunit.client.0.smithi107.stdout:2/635: rmdir d1/d1a/d34/d4f 39 2024-03-11T19:56:20.069 INFO:tasks.workunit.client.0.smithi107.stdout:2/636: truncate d1/d1a/d1e/d26/fb2 263838 0 2024-03-11T19:56:20.070 INFO:tasks.workunit.client.0.smithi107.stdout:3/801: link d1/d10/d13/d4a/d4c/c80 d1/d10/d21/cfe 0 2024-03-11T19:56:20.070 INFO:tasks.workunit.client.0.smithi107.stdout:2/637: mkdir d1/d1a/d1e/d26/dd6 0 2024-03-11T19:56:20.071 INFO:tasks.workunit.client.0.smithi107.stdout:2/638: dread - d1/d38/fc8 zero size 2024-03-11T19:56:20.071 INFO:tasks.workunit.client.0.smithi107.stdout:3/802: unlink d1/d10/f6f 0 2024-03-11T19:56:20.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/803: link d1/d10/d2a/f38 d1/d10/d13/d6c/d5d/fff 0 2024-03-11T19:56:20.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/804: symlink d1/d10/d2a/d7e/da0/l100 0 2024-03-11T19:56:20.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/805: stat d1/d10/d21/cfe 0 2024-03-11T19:56:20.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/639: dread d1/d21/d35/f5c [0,4194304] 0 2024-03-11T19:56:20.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/683: dwrite d4/d7/d1f/d2b/d19/d75/fb2 [0,4194304] 0 2024-03-11T19:56:20.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/684: truncate d4/d7/db/d5e/fcc 1040155 0 2024-03-11T19:56:20.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/640: mkdir d1/d38/d75/dd7 0 2024-03-11T19:56:20.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/641: fdatasync d1/d21/f28 0 2024-03-11T19:56:20.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/685: mkdir d4/d7/db/dde 0 2024-03-11T19:56:20.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/642: mknod d1/d1a/d1e/cd8 0 2024-03-11T19:56:20.091 INFO:tasks.workunit.client.0.smithi107.stdout:5/778: dwrite da/d31/d42/f28 [4194304,4194304] 0 2024-03-11T19:56:20.091 INFO:tasks.workunit.client.0.smithi107.stdout:5/779: chown da/d14/d18/c67 1 1 2024-03-11T19:56:20.092 INFO:tasks.workunit.client.0.smithi107.stdout:5/780: mknod da/d14/d38/d59/d5f/dc9/cf3 0 2024-03-11T19:56:20.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/686: link d4/d7/d2f/d3c/fc6 d4/d7/d1f/d2b/d19/d75/d8e/da5/fdf 0 2024-03-11T19:56:20.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/687: fsync d4/d7/d2f/d3c/fdc 0 2024-03-11T19:56:20.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/579: dwrite dd/de/d1b/f1d [4194304,4194304] 0 2024-03-11T19:56:20.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/580: write dd/d13/d18/f50 [3219523,31861] 0 2024-03-11T19:56:20.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/581: stat dd/de/d1b/d2c/c63 0 2024-03-11T19:56:20.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/756: dread d6/da/d16/d5c/d7d/d1b/f1c [0,4194304] 0 2024-03-11T19:56:20.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/757: write d6/da/d16/d5c/d7d/d1b/f1c [3372076,68677] 0 2024-03-11T19:56:20.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/758: chown d6/d9/cb4 525392355 1 2024-03-11T19:56:20.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/759: chown d6/d9/d57/fa1 439479558 1 2024-03-11T19:56:20.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/688: creat d4/d7/d1f/d2b/d93/fe0 x:0 0 0 2024-03-11T19:56:20.110 INFO:tasks.workunit.client.0.smithi107.stdout:7/760: symlink d6/da/d16/d5c/led 0 2024-03-11T19:56:20.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/761: mkdir d6/da/d49/dee 0 2024-03-11T19:56:20.132 INFO:tasks.workunit.client.0.smithi107.stdout:8/582: dread dd/de/d1b/d20/d5e/fa1 [0,4194304] 0 2024-03-11T19:56:20.132 INFO:tasks.workunit.client.0.smithi107.stdout:3/806: dwrite d1/d10/d13/d6c/d2d/d4d/d91/da5/fd5 [8388608,4194304] 0 2024-03-11T19:56:20.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/583: unlink dd/d13/d18/f50 0 2024-03-11T19:56:20.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/584: fdatasync dd/de/d49/f2b 0 2024-03-11T19:56:20.136 INFO:tasks.workunit.client.0.smithi107.stdout:8/585: mknod dd/de/d1b/d4b/ca7 0 2024-03-11T19:56:20.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/807: mknod d1/d10/d13/d6c/d5d/d76/c101 0 2024-03-11T19:56:20.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/808: getdents d1/d10/d13/d6c/d2d/d4d/d91/df0 0 2024-03-11T19:56:20.140 INFO:tasks.workunit.client.0.smithi107.stdout:3/809: rmdir d1/d10/d13/d4a/d4c/d49/dca/df9 0 2024-03-11T19:56:20.140 INFO:tasks.workunit.client.0.smithi107.stdout:3/810: chown d1/d10/d13/cf4 205231 1 2024-03-11T19:56:20.141 INFO:tasks.workunit.client.0.smithi107.stdout:8/586: mknod dd/ca8 0 2024-03-11T19:56:20.141 INFO:tasks.workunit.client.0.smithi107.stdout:8/587: dread - dd/de/d1b/d20/d56/d87/f8f zero size 2024-03-11T19:56:20.141 INFO:tasks.workunit.client.0.smithi107.stdout:8/588: write dd/de/d1b/d20/d56/d87/f8f [409960,8363] 0 2024-03-11T19:56:20.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/589: creat dd/de/d1b/d20/d5e/fa9 x:0 0 0 2024-03-11T19:56:20.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/590: dread - dd/de/d49/d46/fa5 zero size 2024-03-11T19:56:20.147 INFO:tasks.workunit.client.0.smithi107.stdout:5/781: rename da/d14/d38/d46/faf to da/d14/d18/d1e/d9b/d73/ff4 0 2024-03-11T19:56:20.147 INFO:tasks.workunit.client.0.smithi107.stdout:5/782: fdatasync da/d14/d18/d1e/d9b/d73/fd6 0 2024-03-11T19:56:20.147 INFO:tasks.workunit.client.0.smithi107.stdout:5/783: chown da/d14/d38/d59/d5f/c70 27234 1 2024-03-11T19:56:20.148 INFO:tasks.workunit.client.0.smithi107.stdout:5/784: fdatasync da/d14/d38/f87 0 2024-03-11T19:56:20.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/591: symlink dd/laa 0 2024-03-11T19:56:20.154 INFO:tasks.workunit.client.0.smithi107.stdout:7/762: rename d6/da/fbb to d6/d9/d57/d65/fef 0 2024-03-11T19:56:20.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/763: unlink d6/da/f8f 0 2024-03-11T19:56:20.157 INFO:tasks.workunit.client.0.smithi107.stdout:7/764: mkdir d6/da/d16/d5c/d7d/df0 0 2024-03-11T19:56:20.157 INFO:tasks.workunit.client.0.smithi107.stdout:7/765: fsync d6/d9/f4f 0 2024-03-11T19:56:20.166 INFO:tasks.workunit.client.0.smithi107.stdout:2/643: dwrite d1/d21/f96 [0,4194304] 0 2024-03-11T19:56:20.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/785: dread da/f85 [0,4194304] 0 2024-03-11T19:56:20.167 INFO:tasks.workunit.client.0.smithi107.stdout:3/811: dread d1/dbb/de3/dfd/fb8 [0,4194304] 0 2024-03-11T19:56:20.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/786: write da/d14/d43/d9e/dad/fb0 [244653,38679] 0 2024-03-11T19:56:20.167 INFO:tasks.workunit.client.0.smithi107.stdout:3/812: read d1/d10/d13/f2c [2233205,116899] 0 2024-03-11T19:56:20.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/592: rename dd/de/d1b/d20/d5e/c7c to dd/de/d1b/d20/d56/da4/cab 0 2024-03-11T19:56:20.178 INFO:tasks.workunit.client.0.smithi107.stdout:8/593: mkdir dd/dac 0 2024-03-11T19:56:20.180 INFO:tasks.workunit.client.0.smithi107.stdout:8/594: rename dd/de/d1b/d20/f68 to dd/de/d49/d46/fad 0 2024-03-11T19:56:20.186 INFO:tasks.workunit.client.0.smithi107.stdout:7/766: dread d6/da/d16/f6f [0,4194304] 0 2024-03-11T19:56:20.186 INFO:tasks.workunit.client.0.smithi107.stdout:7/767: chown d6/da/d16/d5c/c9a 89 1 2024-03-11T19:56:20.188 INFO:tasks.workunit.client.0.smithi107.stdout:7/768: link d6/f2f d6/d9/d57/da2/ff1 0 2024-03-11T19:56:20.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/769: stat d6/c66 0 2024-03-11T19:56:20.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/787: link da/d14/d18/d6d/lb5 da/d14/d18/d44/da5/lf5 0 2024-03-11T19:56:20.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/788: chown da/d14/d38/f87 1039116 1 2024-03-11T19:56:20.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/770: write d6/f2f [5742291,67397] 0 2024-03-11T19:56:20.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/771: creat d6/da/d16/d5c/d7d/d60/ff2 x:0 0 0 2024-03-11T19:56:20.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/595: dread dd/d13/d18/f1a [4194304,4194304] 0 2024-03-11T19:56:20.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/789: rename da/d14/d18/d1e/d9b/d73/l7d to da/d14/d18/db2/lf6 0 2024-03-11T19:56:20.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/790: chown da/d14/d43/cd1 9304392 1 2024-03-11T19:56:20.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/791: truncate f4 4428540 0 2024-03-11T19:56:20.210 INFO:tasks.workunit.client.0.smithi107.stdout:5/792: mkdir da/d14/df7 0 2024-03-11T19:56:20.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/793: mkdir da/d14/d18/df8 0 2024-03-11T19:56:20.214 INFO:tasks.workunit.client.0.smithi107.stdout:5/794: rmdir da/d14/d18/df8 0 2024-03-11T19:56:20.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/677: sync 2024-03-11T19:56:20.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/686: sync 2024-03-11T19:56:20.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/813: dwrite d1/d10/d13/d4a/d4c/f2f [4194304,4194304] 0 2024-03-11T19:56:20.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/687: mknod de/d11/d23/d32/d34/d39/d55/cce 0 2024-03-11T19:56:20.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/678: mknod d4/d9/d9f/da7/cee 0 2024-03-11T19:56:20.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/814: getdents d1/d10/d13/d6c/d2d/d4d/d91/dcc 0 2024-03-11T19:56:20.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/688: symlink de/d11/d74/lcf 0 2024-03-11T19:56:20.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/689: stat de/d11/d23/d26/l94 0 2024-03-11T19:56:20.239 INFO:tasks.workunit.client.0.smithi107.stdout:9/679: symlink d4/d1d/d1f/lef 0 2024-03-11T19:56:20.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/690: creat de/d11/d23/d24/da4/dae/fd0 x:0 0 0 2024-03-11T19:56:20.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/815: getdents d1/d10/d4e/d73 0 2024-03-11T19:56:20.242 INFO:tasks.workunit.client.0.smithi107.stdout:9/680: creat d4/d1d/d1f/ff0 x:0 0 0 2024-03-11T19:56:20.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/816: link d1/d10/d13/d6c/d5d/d76/fc8 d1/d10/d13/d6c/d2d/d4d/d91/dcd/f102 0 2024-03-11T19:56:20.245 INFO:tasks.workunit.client.0.smithi107.stdout:9/681: mkdir d4/d1c/d37/dd9/de9/df1 0 2024-03-11T19:56:20.245 INFO:tasks.workunit.client.0.smithi107.stdout:9/682: write d4/d9/dae/d6d/db0/fc2 [1484279,48270] 0 2024-03-11T19:56:20.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/596: dwrite dd/de/d1b/d2c/f2d [0,4194304] 0 2024-03-11T19:56:20.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/597: write dd/f21 [1255458,59677] 0 2024-03-11T19:56:20.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/772: dwrite d6/d21/f41 [4194304,4194304] 0 2024-03-11T19:56:20.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/683: rename d4/d9/d9f/da7 to d4/d1d/d1f/d28/d5d/df2 0 2024-03-11T19:56:20.251 INFO:tasks.workunit.client.0.smithi107.stdout:9/684: readlink d4/d1d/d1f/d28/l66 0 2024-03-11T19:56:20.251 INFO:tasks.workunit.client.0.smithi107.stdout:9/685: chown d4/l2b 8268 1 2024-03-11T19:56:20.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/686: creat d4/d1c/d37/ff3 x:0 0 0 2024-03-11T19:56:20.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/691: dread de/d11/d23/d26/f29 [0,4194304] 0 2024-03-11T19:56:20.266 INFO:tasks.workunit.client.0.smithi107.stdout:0/692: mkdir de/d11/d23/d26/dbf/dd1 0 2024-03-11T19:56:20.266 INFO:tasks.workunit.client.0.smithi107.stdout:0/693: readlink de/d11/d74/l96 0 2024-03-11T19:56:20.267 INFO:tasks.workunit.client.0.smithi107.stdout:3/817: dread d1/d10/d13/d6c/f32 [0,4194304] 0 2024-03-11T19:56:20.268 INFO:tasks.workunit.client.0.smithi107.stdout:3/818: truncate d1/d10/d26/f58 1111147 0 2024-03-11T19:56:20.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/819: readlink d1/d10/d26/lab 0 2024-03-11T19:56:20.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/820: symlink d1/d10/d2a/d7e/d8b/l103 0 2024-03-11T19:56:20.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/821: fdatasync d1/d10/d13/d4a/d4c/d49/f54 0 2024-03-11T19:56:20.279 INFO:tasks.workunit.client.0.smithi107.stdout:3/822: rename d1/d10/d13/d6c/f44 to d1/d10/d21/d43/d4f/f104 0 2024-03-11T19:56:20.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/694: read de/d11/d23/d24/da4/fbd [1723460,27296] 0 2024-03-11T19:56:20.287 INFO:tasks.workunit.client.0.smithi107.stdout:0/695: mkdir de/d11/d23/d24/d83/dc4/dd2 0 2024-03-11T19:56:20.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/773: dread d6/da/d16/d91/dc8/d80/fb0 [0,4194304] 0 2024-03-11T19:56:20.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/774: rename d6/d9/f30 to d6/d9/dae/ff3 0 2024-03-11T19:56:20.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/775: rename d6/da/f45 to d6/da/ff4 0 2024-03-11T19:56:20.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/776: truncate d6/da/d16/d5c/d7d/d60/fbe 951324 0 2024-03-11T19:56:20.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/598: dwrite dd/f99 [0,4194304] 0 2024-03-11T19:56:20.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/687: dread d4/d1d/d1f/d28/d2c/f5b [0,4194304] 0 2024-03-11T19:56:20.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/688: truncate d4/d9/dae/f94 406207 0 2024-03-11T19:56:20.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/689: write d4/d9/dae/d6d/db0/fc2 [2867736,15201] 0 2024-03-11T19:56:20.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/690: getdents d4/d1d/d1f/d28/d5d/df2 0 2024-03-11T19:56:20.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/823: write d1/d10/d13/d6c/d2d/d4d/d91/da5/fd5 [5453506,24886] 0 2024-03-11T19:56:20.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/691: rename d4/d9/dae/d6d/db0/cb6 to d4/d1c/d47/cf4 0 2024-03-11T19:56:20.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/692: mknod d4/d1c/d37/dd9/de9/cf5 0 2024-03-11T19:56:20.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/693: rmdir d4/d1d/d5c 39 2024-03-11T19:56:20.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/696: read de/d11/d23/d24/d4f/f78 [1310795,84623] 0 2024-03-11T19:56:20.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/697: mkdir de/d11/d74/da2/dd3 0 2024-03-11T19:56:20.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/599: read dd/d16/f1e [928799,33786] 0 2024-03-11T19:56:20.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/600: chown dd/de/c11 6543947 1 2024-03-11T19:56:20.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/601: creat dd/de/d49/d7e/fae x:0 0 0 2024-03-11T19:56:20.329 INFO:tasks.workunit.client.0.smithi107.stdout:3/824: write d1/d10/d13/d6c/d2d/d4d/d91/dcd/f102 [2250904,101772] 0 2024-03-11T19:56:20.348 INFO:tasks.workunit.client.0.smithi107.stdout:0/698: rename de/d11/d23/d26 to de/d11/d74/da2/dd3/dd4 0 2024-03-11T19:56:20.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/699: chown de/l68 845633 1 2024-03-11T19:56:20.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/700: dread - de/d11/d74/fbc zero size 2024-03-11T19:56:20.351 INFO:tasks.workunit.client.0.smithi107.stdout:0/701: symlink de/d11/d23/d24/ld5 0 2024-03-11T19:56:20.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/602: read dd/de/d49/f2b [3706258,45990] 0 2024-03-11T19:56:20.357 INFO:tasks.workunit.client.0.smithi107.stdout:8/603: symlink dd/de/d1b/d4b/laf 0 2024-03-11T19:56:20.358 INFO:tasks.workunit.client.0.smithi107.stdout:9/694: dwrite d4/d1c/d37/f50 [0,4194304] 0 2024-03-11T19:56:20.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/604: creat dd/de/d24/d28/d47/fb0 x:0 0 0 2024-03-11T19:56:20.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/605: creat dd/de/d24/d28/d47/fb1 x:0 0 0 2024-03-11T19:56:20.363 INFO:tasks.workunit.client.0.smithi107.stdout:9/695: symlink d4/d9/d9f/dd8/lf6 0 2024-03-11T19:56:20.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/777: dwrite d6/da/d16/d5c/d7d/f5e [0,4194304] 0 2024-03-11T19:56:20.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/778: write d6/da/d16/fbf [165864,66175] 0 2024-03-11T19:56:20.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/779: creat d6/d9/d57/da2/ff5 x:0 0 0 2024-03-11T19:56:20.390 INFO:tasks.workunit.client.0.smithi107.stdout:3/825: dwrite d1/d10/d21/d43/d48/d98/fa8 [0,4194304] 0 2024-03-11T19:56:20.390 INFO:tasks.workunit.client.0.smithi107.stdout:3/826: read - d1/d10/d26/d17/fae zero size 2024-03-11T19:56:20.390 INFO:tasks.workunit.client.0.smithi107.stdout:3/827: readlink d1/d10/d21/d43/d48/d98/le7 0 2024-03-11T19:56:20.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/828: symlink d1/d10/d21/d43/l105 0 2024-03-11T19:56:20.394 INFO:tasks.workunit.client.0.smithi107.stdout:3/829: link d1/d10/d13/d6c/d5d/f94 d1/d10/d13/d6c/d2d/d4d/d91/f106 0 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/632: sync 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/689: sync 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/714: sync 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/795: sync 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/606: rename dd/d16/c2a to dd/de/d24/d28/d47/cb2 0 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/690: readlink d4/d7/d1f/l39 0 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/691: stat d4/d7/d1f/d2b/c71 0 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/692: chown d4/d7/d1f/d2b/d24 1111 1 2024-03-11T19:56:20.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/607: write dd/de/d49/d7e/fae [827433,121327] 0 2024-03-11T19:56:20.397 INFO:tasks.workunit.client.0.smithi107.stdout:3/830: link d1/d10/d13/d6c/d2d/d4d/ffb d1/d10/d13/d6c/d2d/d4d/d91/f107 0 2024-03-11T19:56:20.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/796: creat da/d14/d38/dd0/ff9 x:0 0 0 2024-03-11T19:56:20.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/644: sync 2024-03-11T19:56:20.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/696: write d4/d9/f86 [2803146,24154] 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:8/608: getdents dd/de/d24/d28/d47 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/780: rename d6/da/c33 to d6/da/d16/cf6 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/831: creat d1/d10/d4e/d73/d8d/d9c/f108 x:0 0 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/832: chown d1/d10/d13/f61 163210 1 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:5/797: mknod da/d6e/cfa 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:5/798: write da/d14/d18/d44/da5/fa8 [154033,24185] 0 2024-03-11T19:56:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:9/697: creat d4/d9/dae/d6d/ff7 x:0 0 0 2024-03-11T19:56:20.402 INFO:tasks.workunit.client.0.smithi107.stdout:1/715: unlink d6/d7e/d81/d27/d2c/d32/f7f 0 2024-03-11T19:56:20.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/633: truncate d10/fac 857407 0 2024-03-11T19:56:20.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/781: rmdir d6/da/d16/d91 39 2024-03-11T19:56:20.403 INFO:tasks.workunit.client.0.smithi107.stdout:5/799: truncate da/d14/d18/d6d/fa7 626701 0 2024-03-11T19:56:20.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/716: rename d6/d7e/d81/d27/l3b to d6/d7e/da5/le8 0 2024-03-11T19:56:20.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/717: chown d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62 235408665 1 2024-03-11T19:56:20.403 INFO:tasks.workunit.client.0.smithi107.stdout:9/698: mkdir d4/d1c/d37/dd9/de9/df8 0 2024-03-11T19:56:20.404 INFO:tasks.workunit.client.0.smithi107.stdout:1/718: dread - d6/d5b/fd5 zero size 2024-03-11T19:56:20.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/699: stat d4/d1d/d1f/d28/d2c/d96/la0 0 2024-03-11T19:56:20.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/634: truncate d10/d16/d1f/f25 914255 0 2024-03-11T19:56:20.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/635: fdatasync d10/d3d/f98 0 2024-03-11T19:56:20.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/782: creat d6/d9/d4e/ff7 x:0 0 0 2024-03-11T19:56:20.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/700: unlink d4/lc 0 2024-03-11T19:56:20.409 INFO:tasks.workunit.client.0.smithi107.stdout:7/783: write d6/fd4 [813070,71696] 0 2024-03-11T19:56:20.412 INFO:tasks.workunit.client.0.smithi107.stdout:9/701: creat d4/d1d/d1f/d28/d2c/ff9 x:0 0 0 2024-03-11T19:56:20.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/693: write d4/fbe [1103055,94614] 0 2024-03-11T19:56:20.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/694: write d4/d7/da0/fb4 [789506,74142] 0 2024-03-11T19:56:20.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/784: read d6/da/d16/d5c/d7d/d60/fba [324389,23852] 0 2024-03-11T19:56:20.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/695: stat d4/d7/d2f/d33/d7b/lb6 0 2024-03-11T19:56:20.422 INFO:tasks.workunit.client.0.smithi107.stdout:7/785: link d6/d9/f8c d6/da/d16/d5c/d7d/d60/dd2/de8/ff8 0 2024-03-11T19:56:20.422 INFO:tasks.workunit.client.0.smithi107.stdout:7/786: write d6/d9/d57/da2/ff1 [6543308,71474] 0 2024-03-11T19:56:20.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/696: link d4/d7/db/d5c/l72 d4/d7/da0/le1 0 2024-03-11T19:56:20.427 INFO:tasks.workunit.client.0.smithi107.stdout:4/697: unlink d4/d7/d1f/d2b/d24/d26/d29/fa4 0 2024-03-11T19:56:20.430 INFO:tasks.workunit.client.0.smithi107.stdout:4/698: truncate d4/d7/d1f/d2b/d24/d26/d29/f3f 2474107 0 2024-03-11T19:56:20.431 INFO:tasks.workunit.client.0.smithi107.stdout:4/699: dread - d4/d7/d1f/d2b/d19/d75/d8e/da5/fdf zero size 2024-03-11T19:56:20.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/702: getdents d4/d9/d9f/dd8 0 2024-03-11T19:56:20.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/703: creat d4/d9/dae/ffa x:0 0 0 2024-03-11T19:56:20.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/704: chown d4/d1d/d1f/d28/d2c/f62 947083 1 2024-03-11T19:56:20.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/705: stat d4/d1c/d47/d8a 0 2024-03-11T19:56:20.434 INFO:tasks.workunit.client.0.smithi107.stdout:1/719: write d6/d7e/d81/d27/d2c/f30 [2696554,57228] 0 2024-03-11T19:56:20.434 INFO:tasks.workunit.client.0.smithi107.stdout:9/706: link d4/d1d/d1f/d28/d2c/c7b d4/d1c/d37/dd9/de9/cfb 0 2024-03-11T19:56:20.451 INFO:tasks.workunit.client.0.smithi107.stdout:3/833: dwrite d1/d10/d4e/d73/d8d/d9c/f108 [0,4194304] 0 2024-03-11T19:56:20.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/800: read da/d14/d38/d59/f64 [1159116,27870] 0 2024-03-11T19:56:20.475 INFO:tasks.workunit.client.0.smithi107.stdout:7/787: write d6/da/d16/d5c/d7d/d60/f93 [2956828,2925] 0 2024-03-11T19:56:20.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/788: creat d6/da/d16/d5c/ff9 x:0 0 0 2024-03-11T19:56:20.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/789: creat d6/da/d49/ffa x:0 0 0 2024-03-11T19:56:20.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/790: link d6/lad d6/da/d16/d5c/d7d/d60/d61/lfb 0 2024-03-11T19:56:20.497 INFO:tasks.workunit.client.0.smithi107.stdout:2/645: dwrite d1/d21/d35/fc2 [0,4194304] 0 2024-03-11T19:56:20.497 INFO:tasks.workunit.client.0.smithi107.stdout:6/636: dwrite d10/d19/fa6 [0,4194304] 0 2024-03-11T19:56:20.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/834: dwrite d1/d10/d13/d6c/d2d/d4d/f71 [0,4194304] 0 2024-03-11T19:56:20.506 INFO:tasks.workunit.client.0.smithi107.stdout:3/835: symlink d1/d10/d26/l109 0 2024-03-11T19:56:20.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/609: dwrite dd/f99 [4194304,4194304] 0 2024-03-11T19:56:20.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/836: rename d1/d10/d21/d43/d48/d98/l9a to d1/d10/d13/da1/l10a 0 2024-03-11T19:56:20.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/837: fsync d1/d10/d13/d4a/d4c/d49/fb5 0 2024-03-11T19:56:20.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/838: stat d1/d10/d13/d6c/d5d/d76/c101 0 2024-03-11T19:56:20.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/720: write d6/d28/dc4/f91 [562071,129826] 0 2024-03-11T19:56:20.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/637: creat d10/d16/d24/fd4 x:0 0 0 2024-03-11T19:56:20.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/610: rename dd/d16/f1e to dd/d13/d32/d51/fb3 0 2024-03-11T19:56:20.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/611: dread - dd/d13/d18/f9d zero size 2024-03-11T19:56:20.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/612: truncate dd/de/d24/d28/d47/fb1 907010 0 2024-03-11T19:56:20.513 INFO:tasks.workunit.client.0.smithi107.stdout:1/721: mknod d6/d7e/d81/d27/d2c/d72/dc3/dd4/ce9 0 2024-03-11T19:56:20.513 INFO:tasks.workunit.client.0.smithi107.stdout:1/722: write d6/d5b/f78 [945898,125021] 0 2024-03-11T19:56:20.517 INFO:tasks.workunit.client.0.smithi107.stdout:3/839: mkdir d1/d10/d4e/d10b 0 2024-03-11T19:56:20.518 INFO:tasks.workunit.client.0.smithi107.stdout:3/840: chown d1/d10/d2a/d7e 43 1 2024-03-11T19:56:20.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/723: mknod d6/d7e/da5/cea 0 2024-03-11T19:56:20.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/724: chown d6/d7e/d81/l52 1648 1 2024-03-11T19:56:20.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/841: symlink d1/d10/d4e/d10b/l10c 0 2024-03-11T19:56:20.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/842: fsync d1/d10/d13/d6c/d2d/d4d/d91/da5/fd5 0 2024-03-11T19:56:20.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/843: creat d1/d10/d21/d43/d48/db0/f10d x:0 0 0 2024-03-11T19:56:20.526 INFO:tasks.workunit.client.0.smithi107.stdout:8/613: dread dd/de/d1b/d20/d5e/f65 [0,4194304] 0 2024-03-11T19:56:20.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/646: dread d1/d1a/d1e/d26/f68 [0,4194304] 0 2024-03-11T19:56:20.526 INFO:tasks.workunit.client.0.smithi107.stdout:7/791: dwrite d6/f7e [4194304,4194304] 0 2024-03-11T19:56:20.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/614: creat dd/fb4 x:0 0 0 2024-03-11T19:56:20.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/792: write d6/d9/d4e/fa4 [967414,42084] 0 2024-03-11T19:56:20.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/700: dwrite d4/d7/d1f/d2b/d19/d9a/fcd [0,4194304] 0 2024-03-11T19:56:20.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/707: dread d4/d1d/d1f/d28/d2c/fa5 [0,4194304] 0 2024-03-11T19:56:20.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/708: chown d4/d9/d4d/fd6 1957741 1 2024-03-11T19:56:20.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/638: dread d10/d19/f31 [0,4194304] 0 2024-03-11T19:56:20.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/801: dwrite da/d31/d42/f1f [0,4194304] 0 2024-03-11T19:56:20.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/802: chown da/d14/d18/d1e/d9b/d72/fed 406 1 2024-03-11T19:56:20.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/803: dread - da/d14/d38/fe9 zero size 2024-03-11T19:56:20.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/804: write f4 [4785618,68524] 0 2024-03-11T19:56:20.552 INFO:tasks.workunit.client.0.smithi107.stdout:1/725: dread d6/d28/f41 [0,4194304] 0 2024-03-11T19:56:20.553 INFO:tasks.workunit.client.0.smithi107.stdout:5/805: read da/fb [20202,72137] 0 2024-03-11T19:56:20.557 INFO:tasks.workunit.client.0.smithi107.stdout:5/806: symlink da/d14/d38/dd0/lfb 0 2024-03-11T19:56:20.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/709: dread d4/d1d/d1f/d28/d2c/f5b [4194304,4194304] 0 2024-03-11T19:56:20.581 INFO:tasks.workunit.client.0.smithi107.stdout:3/844: write d1/f5 [4049193,87938] 0 2024-03-11T19:56:20.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/845: chown d1/d10/d13/da1/l10a 1 1 2024-03-11T19:56:20.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/846: chown d1/d10/d13/d6c/d2d/d4d 1101106 1 2024-03-11T19:56:20.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/847: fdatasync d1/d10/d13/d6c/d2d/d4d/f71 0 2024-03-11T19:56:20.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/848: symlink d1/d10/d13/d6c/d2d/d4d/d91/df0/l10e 0 2024-03-11T19:56:20.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/793: dwrite d6/d9/d57/fa1 [0,4194304] 0 2024-03-11T19:56:20.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/726: write d6/d28/f4b [703382,22038] 0 2024-03-11T19:56:20.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/727: creat d6/d7e/da5/dc7/feb x:0 0 0 2024-03-11T19:56:20.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/728: write d6/d7e/d81/d27/d2c/d32/d58/fd0 [523851,115769] 0 2024-03-11T19:56:20.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/647: dwrite d1/d38/d75/f9a [0,4194304] 0 2024-03-11T19:56:20.590 INFO:tasks.workunit.client.0.smithi107.stdout:1/729: mknod d6/d7e/d81/d27/d2c/d7d/dd8/cec 0 2024-03-11T19:56:20.590 INFO:tasks.workunit.client.0.smithi107.stdout:8/615: link dd/d16/f8e dd/de/d24/fb5 0 2024-03-11T19:56:20.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/701: dwrite d4/d7/d2f/d3c/fc6 [0,4194304] 0 2024-03-11T19:56:20.591 INFO:tasks.workunit.client.0.smithi107.stdout:6/639: dread d10/d16/f84 [0,4194304] 0 2024-03-11T19:56:20.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/702: truncate d4/d7/d1f/d2b/d19/f70 1571367 0 2024-03-11T19:56:20.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/703: creat d4/d7/da0/fe2 x:0 0 0 2024-03-11T19:56:20.592 INFO:tasks.workunit.client.0.smithi107.stdout:6/640: symlink d10/d19/d38/ld5 0 2024-03-11T19:56:20.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/648: unlink d1/d21/d35/d67/f8c 0 2024-03-11T19:56:20.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/641: creat d10/d19/d3b/d70/dbb/fd6 x:0 0 0 2024-03-11T19:56:20.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/649: write d1/d1a/d34/f42 [1834768,1349] 0 2024-03-11T19:56:20.594 INFO:tasks.workunit.client.0.smithi107.stdout:8/616: creat dd/fb6 x:0 0 0 2024-03-11T19:56:20.595 INFO:tasks.workunit.client.0.smithi107.stdout:3/849: write d1/d10/d2a/d7e/d8b/f97 [4200111,18078] 0 2024-03-11T19:56:20.595 INFO:tasks.workunit.client.0.smithi107.stdout:3/850: truncate d1/d10/d21/d43/d48/f69 2656800 0 2024-03-11T19:56:20.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/851: creat d1/d10/d13/d4a/f10f x:0 0 0 2024-03-11T19:56:20.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/642: symlink d10/ld7 0 2024-03-11T19:56:20.598 INFO:tasks.workunit.client.0.smithi107.stdout:3/852: rename d1/d10/d13/d6c/l24 to d1/d45/d85/l110 0 2024-03-11T19:56:20.598 INFO:tasks.workunit.client.0.smithi107.stdout:3/853: stat d1/d10/d4e/d73/d8d/f95 0 2024-03-11T19:56:20.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/854: mknod d1/d10/d13/c111 0 2024-03-11T19:56:20.601 INFO:tasks.workunit.client.0.smithi107.stdout:3/855: truncate d1/d10/d21/d43/d48/d98/fa8 2246359 0 2024-03-11T19:56:20.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/710: write f1 [3620546,4366] 0 2024-03-11T19:56:20.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/711: dread - d4/d1c/d37/f75 zero size 2024-03-11T19:56:20.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/712: fsync d4/f54 0 2024-03-11T19:56:20.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/713: symlink d4/d1c/d37/dd9/de9/df1/lfc 0 2024-03-11T19:56:20.615 INFO:tasks.workunit.client.0.smithi107.stdout:9/714: mkdir d4/d1d/d4b/dfd 0 2024-03-11T19:56:20.621 INFO:tasks.workunit.client.0.smithi107.stdout:0/702: sync 2024-03-11T19:56:20.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/794: write d6/da/d16/f3f [1372690,22887] 0 2024-03-11T19:56:20.650 INFO:tasks.workunit.client.0.smithi107.stdout:9/715: fdatasync f1 0 2024-03-11T19:56:20.650 INFO:tasks.workunit.client.0.smithi107.stdout:9/716: write d4/d9/dae/d6d/db0/fc2 [2115456,24302] 0 2024-03-11T19:56:20.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/617: dwrite dd/de/f4e [0,4194304] 0 2024-03-11T19:56:20.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/704: write d4/d7/f8 [55657,102821] 0 2024-03-11T19:56:20.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/705: fdatasync d4/d7/d1f/d2b/d19/fa7 0 2024-03-11T19:56:20.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/618: creat dd/de/d1b/d4b/fb7 x:0 0 0 2024-03-11T19:56:20.655 INFO:tasks.workunit.client.0.smithi107.stdout:4/706: getdents d4/d7/d2f/d33/d7b 0 2024-03-11T19:56:20.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/707: rename d4/fc to d4/d7/d1f/d2b/d19/d75/fe3 0 2024-03-11T19:56:20.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/708: fsync d4/d7/d2f/d3b/f7e 0 2024-03-11T19:56:20.665 INFO:tasks.workunit.client.0.smithi107.stdout:7/795: fsync d6/d9/d57/da2/ff1 0 2024-03-11T19:56:20.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/703: write de/d11/d23/d32/d34/d39/d3a/d72/f7d [356910,46116] 0 2024-03-11T19:56:20.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/704: dread - de/d11/d23/d32/d34/d39/d3a/d72/fb2 zero size 2024-03-11T19:56:20.668 INFO:tasks.workunit.client.0.smithi107.stdout:7/796: getdents d6/d9/dae 0 2024-03-11T19:56:20.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/705: truncate de/f13 385400 0 2024-03-11T19:56:20.670 INFO:tasks.workunit.client.0.smithi107.stdout:5/807: dwrite da/d14/d38/d59/dbd/fdd [0,4194304] 0 2024-03-11T19:56:20.674 INFO:tasks.workunit.client.0.smithi107.stdout:2/650: dwrite d1/f2b [0,4194304] 0 2024-03-11T19:56:20.674 INFO:tasks.workunit.client.0.smithi107.stdout:2/651: fdatasync d1/d38/d75/d76/fc7 0 2024-03-11T19:56:20.677 INFO:tasks.workunit.client.0.smithi107.stdout:7/797: mkdir d6/da/d49/dee/dfc 0 2024-03-11T19:56:20.678 INFO:tasks.workunit.client.0.smithi107.stdout:7/798: fsync d6/da/d16/d5c/d7d/d60/fba 0 2024-03-11T19:56:20.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/706: creat de/d11/d74/da2/dd3/dd4/dbf/dd1/fd6 x:0 0 0 2024-03-11T19:56:20.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/808: creat da/d14/d18/d1e/d9b/db3/ffc x:0 0 0 2024-03-11T19:56:20.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/652: creat d1/d1a/d34/d4f/d63/db6/fd9 x:0 0 0 2024-03-11T19:56:20.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/799: rmdir d6/d9 39 2024-03-11T19:56:20.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/653: write d1/d21/d35/d67/fc0 [10288,34685] 0 2024-03-11T19:56:20.689 INFO:tasks.workunit.client.0.smithi107.stdout:2/654: mkdir d1/d21/dda 0 2024-03-11T19:56:20.689 INFO:tasks.workunit.client.0.smithi107.stdout:7/800: rename d6/da/d16/d91/d97 to d6/da/d16/d5c/d7d/d1b/dfd 0 2024-03-11T19:56:20.689 INFO:tasks.workunit.client.0.smithi107.stdout:7/801: symlink d6/d9/d57/da2/lfe 0 2024-03-11T19:56:20.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/643: dwrite d10/d16/d2b/f59 [0,4194304] 0 2024-03-11T19:56:20.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/644: chown d10/d16/d24/d2e/f34 13755 1 2024-03-11T19:56:20.690 INFO:tasks.workunit.client.0.smithi107.stdout:7/802: symlink d6/da/d16/d5c/d7d/d60/dd2/de8/lff 0 2024-03-11T19:56:20.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/803: symlink d6/d9/dae/l100 0 2024-03-11T19:56:20.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/804: dread - d6/da/d4d/fd5 zero size 2024-03-11T19:56:20.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/730: dwrite d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82/f9a [0,4194304] 0 2024-03-11T19:56:20.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/731: chown d6/d7e/d81/d27/d2c/d32/dc1/f96 3844 1 2024-03-11T19:56:20.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/732: symlink d6/d7e/d81/d27/d2c/d7d/led 0 2024-03-11T19:56:20.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/645: truncate d10/d16/d2b/d39/fb3 3788978 0 2024-03-11T19:56:20.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/809: dread da/d14/f79 [0,4194304] 0 2024-03-11T19:56:20.697 INFO:tasks.workunit.client.0.smithi107.stdout:1/733: symlink d6/d7e/da5/lee 0 2024-03-11T19:56:20.697 INFO:tasks.workunit.client.0.smithi107.stdout:1/734: readlink d6/d7e/da5/ldf 0 2024-03-11T19:56:20.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/856: dwrite d1/d10/d13/d6c/d2d/d4d/d91/dcd/f102 [0,4194304] 0 2024-03-11T19:56:20.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/857: stat d1/d10/d4e/d10b 0 2024-03-11T19:56:20.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/858: readlink d1/dbb/de3/lf2 0 2024-03-11T19:56:20.735 INFO:tasks.workunit.client.0.smithi107.stdout:4/709: dwrite d4/d7/d1f/d2b/d24/f3e [0,4194304] 0 2024-03-11T19:56:20.736 INFO:tasks.workunit.client.0.smithi107.stdout:4/710: truncate d4/d7/d1f/d2b/d19/d75/d8e/fbc 98573 0 2024-03-11T19:56:20.736 INFO:tasks.workunit.client.0.smithi107.stdout:4/711: mknod d4/d7/ce4 0 2024-03-11T19:56:20.746 INFO:tasks.workunit.client.0.smithi107.stdout:9/717: dwrite d4/d1d/d1f/ff0 [0,4194304] 0 2024-03-11T19:56:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/655: dread d1/d38/d75/f7d [4194304,4194304] 0 2024-03-11T19:56:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/656: write d1/f10 [264318,42062] 0 2024-03-11T19:56:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:0/707: dread f3 [0,4194304] 0 2024-03-11T19:56:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:0/708: fdatasync f3 0 2024-03-11T19:56:20.760 INFO:tasks.workunit.client.0.smithi107.stdout:9/718: symlink d4/d9/lfe 0 2024-03-11T19:56:20.760 INFO:tasks.workunit.client.0.smithi107.stdout:9/719: stat d4/d1d/d1f/d78/fb2 0 2024-03-11T19:56:20.761 INFO:tasks.workunit.client.0.smithi107.stdout:7/805: dwrite d6/da/d16/d5c/d7d/f5e [0,4194304] 0 2024-03-11T19:56:20.763 INFO:tasks.workunit.client.0.smithi107.stdout:0/709: write de/d58/f7f [295811,68163] 0 2024-03-11T19:56:20.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/657: creat d1/d1a/d34/d4f/d63/fdb x:0 0 0 2024-03-11T19:56:20.766 INFO:tasks.workunit.client.0.smithi107.stdout:9/720: getdents d4/d1c/d47/d8a 0 2024-03-11T19:56:20.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/721: dread - d4/d1d/d5c/d97/faa zero size 2024-03-11T19:56:20.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/859: write d1/d10/d4e/d73/d8d/d9c/f108 [74745,37399] 0 2024-03-11T19:56:20.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/860: creat d1/d10/d13/d6c/d2d/f112 x:0 0 0 2024-03-11T19:56:20.783 INFO:tasks.workunit.client.0.smithi107.stdout:1/735: dwrite d6/fd [0,4194304] 0 2024-03-11T19:56:20.784 INFO:tasks.workunit.client.0.smithi107.stdout:5/810: dwrite da/d14/d38/f3f [0,4194304] 0 2024-03-11T19:56:20.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/811: truncate da/d14/d38/d59/d5f/f80 809637 0 2024-03-11T19:56:20.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/812: dread - da/d14/d18/ff0 zero size 2024-03-11T19:56:20.790 INFO:tasks.workunit.client.0.smithi107.stdout:6/646: dwrite d10/d16/d1f/f79 [0,4194304] 0 2024-03-11T19:56:20.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/712: dwrite d4/d7/d1f/d2b/d24/d26/d29/f32 [0,4194304] 0 2024-03-11T19:56:20.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/713: dread - d4/d7/d1f/d2b/d93/dc3/dd0/d79/faa zero size 2024-03-11T19:56:20.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/714: fdatasync d4/d7/d2f/d33/f66 0 2024-03-11T19:56:20.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/715: truncate d4/d7/d1f/d2b/d19/d75/d8e/da5/fdf 3078943 0 2024-03-11T19:56:20.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/806: dwrite d6/d9/d57/f5a [0,4194304] 0 2024-03-11T19:56:20.811 INFO:tasks.workunit.client.0.smithi107.stdout:4/716: creat d4/d7/d1f/d2b/d19/d75/fe5 x:0 0 0 2024-03-11T19:56:20.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/710: rename de/d11/d74/da2/dd3/dd4/dca to de/d11/d23/d32/d34/d39/d55/d7c/dd7 0 2024-03-11T19:56:20.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/658: rename d1/d1a/d34/d4f/d63/cad to d1/cdc 0 2024-03-11T19:56:20.822 INFO:tasks.workunit.client.0.smithi107.stdout:2/659: rmdir d1/d38 39 2024-03-11T19:56:20.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/711: read f3 [3799426,82182] 0 2024-03-11T19:56:20.834 INFO:tasks.workunit.client.0.smithi107.stdout:9/722: dwrite d4/d1d/d5c/d97/fe2 [0,4194304] 0 2024-03-11T19:56:20.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/736: write d6/f1d [7212783,17912] 0 2024-03-11T19:56:20.837 INFO:tasks.workunit.client.0.smithi107.stdout:2/660: write d1/d21/d35/d67/f6e [541857,106570] 0 2024-03-11T19:56:20.837 INFO:tasks.workunit.client.0.smithi107.stdout:2/661: write d1/d1a/d1e/d41/fcf [768690,123183] 0 2024-03-11T19:56:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/861: dwrite d1/d10/d26/fdb [0,4194304] 0 2024-03-11T19:56:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/862: readlink d1/d10/d4e/lf6 0 2024-03-11T19:56:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/737: mkdir d6/d7e/d81/d27/d2c/d32/dc1/dc9/def 0 2024-03-11T19:56:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/738: write d6/d7e/d81/d27/f7b [1603981,4540] 0 2024-03-11T19:56:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/739: creat d6/d7e/d81/d27/d2c/d72/ff0 x:0 0 0 2024-03-11T19:56:20.840 INFO:tasks.workunit.client.0.smithi107.stdout:1/740: creat d6/d7e/d81/d27/d2c/d7d/dd8/ff1 x:0 0 0 2024-03-11T19:56:20.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/662: creat d1/fdd x:0 0 0 2024-03-11T19:56:20.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/663: fsync d1/d38/d75/d76/f98 0 2024-03-11T19:56:20.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/863: symlink d1/l113 0 2024-03-11T19:56:20.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/864: creat d1/d10/d21/f114 x:0 0 0 2024-03-11T19:56:20.843 INFO:tasks.workunit.client.0.smithi107.stdout:7/807: dread d6/d9/d57/fda [0,4194304] 0 2024-03-11T19:56:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/808: chown d6/d21/l44 1168 1 2024-03-11T19:56:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/809: chown d6/da/d16/d5c/l63 928099 1 2024-03-11T19:56:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/810: read d6/da/d16/d91/dc8/d80/fb0 [3477627,70699] 0 2024-03-11T19:56:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:9/723: creat d4/d9/dae/d6d/db0/fff x:0 0 0 2024-03-11T19:56:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:9/724: fsync d4/d1d/d1f/d78/fc5 0 2024-03-11T19:56:20.849 INFO:tasks.workunit.client.0.smithi107.stdout:2/664: creat d1/d1a/fde x:0 0 0 2024-03-11T19:56:20.849 INFO:tasks.workunit.client.0.smithi107.stdout:2/665: fdatasync d1/d21/f28 0 2024-03-11T19:56:20.850 INFO:tasks.workunit.client.0.smithi107.stdout:7/811: rmdir d6/da/d49/dee 39 2024-03-11T19:56:20.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/741: write d6/d7e/d81/f4e [710014,1192] 0 2024-03-11T19:56:20.850 INFO:tasks.workunit.client.0.smithi107.stdout:9/725: mkdir d4/d1d/d5c/d64/db3/d100 0 2024-03-11T19:56:20.853 INFO:tasks.workunit.client.0.smithi107.stdout:6/647: dwrite d10/d3d/f62 [0,4194304] 0 2024-03-11T19:56:20.859 INFO:tasks.workunit.client.0.smithi107.stdout:2/666: link d1/f36 d1/d1a/d34/d4f/d63/fdf 0 2024-03-11T19:56:20.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/726: creat d4/d9/f101 x:0 0 0 2024-03-11T19:56:20.859 INFO:tasks.workunit.client.0.smithi107.stdout:7/812: truncate d6/f17 1679875 0 2024-03-11T19:56:20.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/813: fsync d6/d9/d57/d65/fef 0 2024-03-11T19:56:20.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/814: chown d6/da/d49 2 1 2024-03-11T19:56:20.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/667: rmdir d1/d1a/d1e/d26 39 2024-03-11T19:56:20.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/668: write d1/d1d/f77 [966991,32520] 0 2024-03-11T19:56:20.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/727: mknod d4/d1d/d1f/d78/c102 0 2024-03-11T19:56:20.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/728: truncate d4/d9/dae/f7d 4223764 0 2024-03-11T19:56:20.863 INFO:tasks.workunit.client.0.smithi107.stdout:9/729: chown d4/d1c/d37/l44 197365 1 2024-03-11T19:56:20.866 INFO:tasks.workunit.client.0.smithi107.stdout:2/669: unlink d1/d1a/d1e/d41/fcf 0 2024-03-11T19:56:20.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/717: dwrite d4/d7/da0/fe2 [0,4194304] 0 2024-03-11T19:56:20.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/815: dwrite d6/d9/d57/fa1 [0,4194304] 0 2024-03-11T19:56:20.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/712: dread de/d11/d23/d24/f69 [0,4194304] 0 2024-03-11T19:56:20.926 INFO:tasks.workunit.client.0.smithi107.stdout:7/816: dwrite d6/da/d16/d5c/d7d/d60/f93 [0,4194304] 0 2024-03-11T19:56:20.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/648: dread d10/d16/d24/f27 [0,4194304] 0 2024-03-11T19:56:20.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/865: dwrite d1/d10/d21/d43/d4f/f92 [0,4194304] 0 2024-03-11T19:56:20.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/866: creat d1/d10/d4e/d73/d8d/d9c/f115 x:0 0 0 2024-03-11T19:56:20.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/867: truncate d1/d10/d4e/d73/fc3 823237 0 2024-03-11T19:56:20.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/868: truncate d1/d10/d21/d43/d48/fe9 679312 0 2024-03-11T19:56:20.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/670: dread d1/d38/d75/f7d [4194304,4194304] 0 2024-03-11T19:56:20.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/718: dwrite d4/d7/d1f/d2b/d19/d75/fb5 [4194304,4194304] 0 2024-03-11T19:56:20.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/671: creat d1/fe0 x:0 0 0 2024-03-11T19:56:20.939 INFO:tasks.workunit.client.0.smithi107.stdout:4/719: truncate d4/d7/d2f/d3c/f73 302157 0 2024-03-11T19:56:20.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/672: link d1/d1a/d34/d4f/lb1 d1/d1a/d34/le1 0 2024-03-11T19:56:20.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/673: fdatasync d1/d1a/d34/d4f/d63/f6f 0 2024-03-11T19:56:20.947 INFO:tasks.workunit.client.0.smithi107.stdout:4/720: dread d4/d7/d1f/d2b/d24/f5b [0,4194304] 0 2024-03-11T19:56:20.948 INFO:tasks.workunit.client.0.smithi107.stdout:4/721: truncate d4/d7/d1f/d2b/d19/d75/d8e/fa8 1215515 0 2024-03-11T19:56:20.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/674: dread d1/d1a/f20 [0,4194304] 0 2024-03-11T19:56:20.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/675: truncate d1/d38/fc8 924366 0 2024-03-11T19:56:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/817: write d6/da/d16/d5c/d7d/fa9 [3237385,48687] 0 2024-03-11T19:56:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:2/676: dread d1/d1a/d1e/d26/f30 [0,4194304] 0 2024-03-11T19:56:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/818: unlink d6/da/d16/d5c/d7d/d1b/c55 0 2024-03-11T19:56:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/819: fsync d6/d9/fa6 0 2024-03-11T19:56:20.975 INFO:tasks.workunit.client.0.smithi107.stdout:2/677: dread d1/f40 [0,4194304] 0 2024-03-11T19:56:20.975 INFO:tasks.workunit.client.0.smithi107.stdout:2/678: creat d1/fe2 x:0 0 0 2024-03-11T19:56:21.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:20 smithi122 ceph-mon[36397]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 483 KiB/s wr, 223 op/s 2024-03-11T19:56:21.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:20 smithi145 ceph-mon[36730]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 483 KiB/s wr, 223 op/s 2024-03-11T19:56:21.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:20 smithi107 ceph-mon[31712]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 483 KiB/s wr, 223 op/s 2024-03-11T19:56:23.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:22 smithi122 ceph-mon[36397]: pgmap v91: 129 pgs: 129 active+clean; 1.8 GiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 252 op/s 2024-03-11T19:56:23.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:22 smithi145 ceph-mon[36730]: pgmap v91: 129 pgs: 129 active+clean; 1.8 GiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 252 op/s 2024-03-11T19:56:23.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:22 smithi107 ceph-mon[31712]: pgmap v91: 129 pgs: 129 active+clean; 1.8 GiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 252 op/s 2024-03-11T19:56:25.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:24 smithi122 ceph-mon[36397]: pgmap v92: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 124 op/s 2024-03-11T19:56:25.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:24 smithi145 ceph-mon[36730]: pgmap v92: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 124 op/s 2024-03-11T19:56:25.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:24 smithi107 ceph-mon[31712]: pgmap v92: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 124 op/s 2024-03-11T19:56:27.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:26 smithi122 ceph-mon[36397]: pgmap v93: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 66 op/s 2024-03-11T19:56:27.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:26 smithi145 ceph-mon[36730]: pgmap v93: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 66 op/s 2024-03-11T19:56:27.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:26 smithi107 ceph-mon[31712]: pgmap v93: 129 pgs: 129 active+clean; 1.9 GiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 66 op/s 2024-03-11T19:56:29.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:28 smithi122 ceph-mon[36397]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 83 op/s 2024-03-11T19:56:29.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:28 smithi145 ceph-mon[36730]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 83 op/s 2024-03-11T19:56:29.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:28 smithi107 ceph-mon[31712]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 83 op/s 2024-03-11T19:56:31.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:30 smithi122 ceph-mon[36397]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 74 op/s 2024-03-11T19:56:31.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:30 smithi145 ceph-mon[36730]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 74 op/s 2024-03-11T19:56:31.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:30 smithi107 ceph-mon[31712]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 74 op/s 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 -- 172.21.15.107:0/1017602462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 msgr2=0x7fc86c101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 --2- 172.21.15.107:0/1017602462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c101780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc860009930 tx=0x7fc86002bc20 comp rx=0 tx=0).stop 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 -- 172.21.15.107:0/1017602462 shutdown_connections 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 --2- 172.21.15.107:0/1017602462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c106470 0x7fc86c10e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 --2- 172.21.15.107:0/1017602462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c101d50 0x7fc86c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 --2- 172.21.15.107:0/1017602462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c101780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 -- 172.21.15.107:0/1017602462 >> 172.21.15.107:0/1017602462 conn(0x7fc86c0fcc00 msgr2=0x7fc86c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 -- 172.21.15.107:0/1017602462 shutdown_connections 2024-03-11T19:56:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.427+0000 7fc873a00640 1 -- 172.21.15.107:0/1017602462 wait complete. 2024-03-11T19:56:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.428+0000 7fc873a00640 1 Processor -- start 2024-03-11T19:56:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 -- start start 2024-03-11T19:56:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c106470 0x7fc86c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 0x7fc86c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1a5f30 con 0x7fc86c073f00 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1a60a0 con 0x7fc86c1013a0 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc873a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1a6260 con 0x7fc86c106470 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc871000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc871000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43642/0 (socket says 172.21.15.107:43642) 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.429+0000 7fc871000640 1 -- 172.21.15.107:0/2932434357 learned_addr learned my addr 172.21.15.107:0/2932434357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc86be00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c106470 0x7fc86c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 0x7fc86c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 -- 172.21.15.107:0/2932434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c106470 msgr2=0x7fc86c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c106470 0x7fc86c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 -- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 msgr2=0x7fc86c1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 0x7fc86c1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 -- 172.21.15.107:0/2932434357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860009560 con 0x7fc86c1013a0 2024-03-11T19:56:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 0x7fc86c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:56:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc871000640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc860009230 tx=0x7fc86002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.430+0000 7fc869600640 1 -- 172.21.15.107:0/2932434357 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860004350 con 0x7fc86c1013a0 2024-03-11T19:56:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.431+0000 7fc869600640 1 -- 172.21.15.107:0/2932434357 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8600044b0 con 0x7fc86c1013a0 2024-03-11T19:56:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.431+0000 7fc869600640 1 -- 172.21.15.107:0/2932434357 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860003d00 con 0x7fc86c1013a0 2024-03-11T19:56:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.431+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c1a64e0 con 0x7fc86c1013a0 2024-03-11T19:56:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.431+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1a69f0 con 0x7fc86c1013a0 2024-03-11T19:56:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 Processor -- start 2024-03-11T19:56:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 -- start start 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790e1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887909c620 con 0x5588790ea330 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887904e8c0 con 0x5588790e1b20 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.435+0000 7fc8767d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887904bdf0 con 0x5588790e0db0 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc869600640 1 -- 172.21.15.107:0/2932434357 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc860004df0 con 0x7fc86c1013a0 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43656/0 (socket says 172.21.15.107:43656) 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 -- 172.21.15.107:0/2099619264 learned_addr learned my addr 172.21.15.107:0/2099619264 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc86be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 -- 172.21.15.107:0/2099619264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 msgr2=0x5588790e1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871000640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790e1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790e1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 -- 172.21.15.107:0/2099619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 msgr2=0x5588790ec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871a00640 1 -- 172.21.15.107:0/2099619264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558878fc8b00 con 0x5588790e1b20 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc86be00640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.436+0000 7fc871000640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790e1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.437+0000 7fc871a00640 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc864009f10 tx=0x7fc86400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb93902bbb70fe51 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.437+0000 7fc85e000640 1 -- 172.21.15.107:0/2099619264 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864003b70 con 0x5588790e1b20 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.437+0000 7fc85e000640 1 -- 172.21.15.107:0/2099619264 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc864003cd0 con 0x5588790e1b20 2024-03-11T19:56:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.437+0000 7fc85e000640 1 -- 172.21.15.107:0/2099619264 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640118d0 con 0x5588790e1b20 2024-03-11T19:56:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.437+0000 7fc869600640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 0x7fc8540a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 -- 172.21.15.107:0/2099619264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 msgr2=0x5588790e0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc864009f10 tx=0x7fc86400f730 comp rx=0 tx=0).stop 2024-03-11T19:56:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc869600640 1 -- 172.21.15.107:0/2932434357 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc8600efb80 con 0x7fc86c1013a0 2024-03-11T19:56:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 -- 172.21.15.107:0/2099619264 shutdown_connections 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc871000640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 0x7fc8540a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790e1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2099619264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e0870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 -- 172.21.15.107:0/2099619264 >> 172.21.15.107:0/2099619264 conn(0x5588790a84b0 msgr2=0x5588790dbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 -- 172.21.15.107:0/2099619264 shutdown_connections 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc8767d6740 1 -- 172.21.15.107:0/2099619264 wait complete. 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.438+0000 7fc871000640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 0x7fc8540a4130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5588790e2280 tx=0x7fc8600dc040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 -- start start 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 Processor -- start 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e0db0 0x5588790e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588790475d0 con 0x5588790e1b20 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x558878fecb50 con 0x5588790e0db0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x558878fec7f0 con 0x5588790ea330 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.439+0000 7fc8767d6740 10 client.0 _ll_register_callbacks cb 0x558879000550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43376/0 (socket says 172.21.15.107:43376) 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc871a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 -- 172.21.15.107:0/3791924800 learned_addr learned my addr 172.21.15.107:0/3791924800 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc871a00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e0db0 0x5588790e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 -- 172.21.15.107:0/3791924800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 msgr2=0x5588790e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 -- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e0db0 msgr2=0x5588790e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e0db0 0x5588790e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 -- 172.21.15.107:0/3791924800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc864009bc0 con 0x5588790e1b20 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc86be00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc858012710 tx=0x7fc858012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858d288c568fb100 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.440+0000 7fc852a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858017070 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc858004050 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558879019cd0 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc858004210 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc8767d6740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558879045c50 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558879019cd0 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.441+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc858003a60 con 0x5588790e1b20 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 20 client.15612 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5588790fa3b0 0x5588790fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 4 client.15612 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:56:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55887903dce0 con 0x5588790fa3b0 2024-03-11T19:56:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc871a00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5588790fa3b0 0x5588790fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc871a00640 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5588790fa3b0 0x5588790fc7d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5949b6ab07c9b09d server_cookie=d725e15993475d8c in_seq=0 out_seq=0 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.442+0000 7fc852a00640 10 client.15612 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.443+0000 7fc852a00640 1 -- 172.21.15.107:0/3791924800 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55887903dce0 con 0x5588790fa3b0 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.443+0000 7fc852a00640 10 client.15612 handle_command_reply: tid=0 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.443+0000 7fc8767d6740 1 client.15612 shutdown 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 msgr2=0x5588790e78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc858012710 tx=0x7fc858012bd0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 shutdown_connections 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5588790fa3b0 0x5588790fc7d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e1b20 0x5588790e78c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 --2- 172.21.15.107:0/3791924800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e0db0 0x5588790e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc852a00640 0 client.15612 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 >> 172.21.15.107:0/3791924800 conn(0x5588790a84b0 msgr2=0x5588790dd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 shutdown_connections 2024-03-11T19:56:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 1 -- 172.21.15.107:0/3791924800 wait complete. 2024-03-11T19:56:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.444+0000 7fc8767d6740 20 client.15612 trim_cache size 0 max 16384 2024-03-11T19:56:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.459+0000 7fc8767d6740 1 Processor -- start 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.459+0000 7fc8767d6740 1 -- start start 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 0x5588790e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558878d8fe60 con 0x5588790e0db0 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887903dd10 con 0x5588790e1b20 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc8767d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588790e2110 con 0x5588790ea330 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc86be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc871a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 0x5588790e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc86be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43682/0 (socket says 172.21.15.107:43682) 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc86be00640 1 -- 172.21.15.107:0/2621896387 learned_addr learned my addr 172.21.15.107:0/2621896387 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.460+0000 7fc871a00640 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 -- 172.21.15.107:0/2621896387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 msgr2=0x5588790e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 -- 172.21.15.107:0/2621896387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 msgr2=0x5588790e78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 0x5588790e78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 -- 172.21.15.107:0/2621896387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558878fe2530 con 0x5588790e1b20 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc871a00640 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 0x5588790e78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc86be00640 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc858009de0 tx=0x7fc8580096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6ceada1dabcda0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc85d600640 1 -- 172.21.15.107:0/2621896387 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858017070 con 0x5588790e1b20 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc85d600640 1 -- 172.21.15.107:0/2621896387 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8580055a0 con 0x5588790e1b20 2024-03-11T19:56:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.461+0000 7fc85d600640 1 -- 172.21.15.107:0/2621896387 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580035f0 con 0x5588790e1b20 2024-03-11T19:56:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 -- 172.21.15.107:0/2621896387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 msgr2=0x5588790e7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc858009de0 tx=0x7fc8580096a0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 -- 172.21.15.107:0/2621896387 shutdown_connections 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790e7e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790ea330 0x5588790e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 --2- 172.21.15.107:0/2621896387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790e0db0 0x5588790e78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 -- 172.21.15.107:0/2621896387 >> 172.21.15.107:0/2621896387 conn(0x5588790a84b0 msgr2=0x5588790db5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 -- 172.21.15.107:0/2621896387 shutdown_connections 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.462+0000 7fc8767d6740 1 -- 172.21.15.107:0/2621896387 wait complete. 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 -- start start 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 Processor -- start 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588790efde0 con 0x5588790ea330 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588790eff50 con 0x5588790e1b20 2024-03-11T19:56:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588790f0110 con 0x5588790e0db0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.463+0000 7fc8767d6740 10 client.0 _ll_register_callbacks cb 0x7fc858009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc871a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc871a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43402/0 (socket says 172.21.15.107:43402) 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc871a00640 1 -- 172.21.15.107:0/4186454921 learned_addr learned my addr 172.21.15.107:0/4186454921 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc871a00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 -- 172.21.15.107:0/4186454921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 msgr2=0x5588790f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 -- 172.21.15.107:0/4186454921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 msgr2=0x5588790ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 -- 172.21.15.107:0/4186454921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558878fc8b00 con 0x5588790e1b20 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc86be00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790f16b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5588790d04b0 tx=0x7fc8580106f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174082413b4d84f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.464+0000 7fc852a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:56:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858017040 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55887906cd30 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc858000e40 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc85800f920 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 10 client.25751 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc858021040 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55887906cd30 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc858030070 con 0x5588790e1b20 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 10 client.25751 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 20 client.25751 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558879151980 0x558879153da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 4 client.25751 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.465+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558878d47600 con 0x558879151980 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.466+0000 7fc871a00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558879151980 0x558879153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.466+0000 7fc871a00640 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558879151980 0x558879153da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6b81024617ff63 server_cookie=3fa62ad0edf21800 in_seq=0 out_seq=0 2024-03-11T19:56:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.466+0000 7fc852a00640 10 client.25751 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:31.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/813: rename da/d14/d18/d44/da5/fa8 to da/d14/d38/d46/ffd 0 2024-03-11T19:56:31.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/869: sync 2024-03-11T19:56:31.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/820: sync 2024-03-11T19:56:31.664 INFO:tasks.workunit.client.0.smithi107.stdout:9/730: sync 2024-03-11T19:56:31.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/814: mknod da/d14/d18/d1e/d9b/d72/cfe 0 2024-03-11T19:56:31.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/815: write da/d14/d96/dc1/fdc [893569,39915] 0 2024-03-11T19:56:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.664+0000 7fc852a00640 1 -- 172.21.15.107:0/4186454921 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558878d47600 con 0x558879151980 2024-03-11T19:56:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.664+0000 7fc852a00640 10 client.25751 handle_command_reply: tid=0 2024-03-11T19:56:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.664+0000 7fc8767d6740 1 client.25751 shutdown 2024-03-11T19:56:31.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/619: sync 2024-03-11T19:56:31.666 INFO:tasks.workunit.client.0.smithi107.stdout:4/722: sync 2024-03-11T19:56:31.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/620: chown dd/d13/d32/f78 42074 1 2024-03-11T19:56:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 msgr2=0x5588790f16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790f16b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5588790d04b0 tx=0x7fc8580106f0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 shutdown_connections 2024-03-11T19:56:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x558879151980 0x558879153da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.668 INFO:tasks.workunit.client.0.smithi107.stdout:7/821: link d6/da/d16/d5c/ff9 d6/da/d16/d5c/d7d/df0/f101 0 2024-03-11T19:56:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc852a00640 0 client.25751 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588790e1b20 0x5588790f16b0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5588790d04b0 tx=0x7fc8580106f0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588790ea330 0x5588790ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 --2- 172.21.15.107:0/4186454921 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5588790e0db0 0x5588790f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.666+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 >> 172.21.15.107:0/4186454921 conn(0x5588790a84b0 msgr2=0x5588790e5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.668+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 shutdown_connections 2024-03-11T19:56:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.668+0000 7fc8767d6740 1 -- 172.21.15.107:0/4186454921 wait complete. 2024-03-11T19:56:31.670 INFO:tasks.workunit.client.0.smithi107.stdout:4/723: truncate d4/d7/d1f/d2b/d24/d26/d29/d36/f8c 1347548 0 2024-03-11T19:56:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.669+0000 7fc8767d6740 20 client.25751 trim_cache size 0 max 16384 2024-03-11T19:56:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:56:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:56:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:56:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:56:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.670+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 msgr2=0x7fc8540a4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.670+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 0x7fc8540a4130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5588790e2280 tx=0x7fc8600dc040 comp rx=0 tx=0).stop 2024-03-11T19:56:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.670+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 msgr2=0x7fc86c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.670+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc860009230 tx=0x7fc86002c640 comp rx=0 tx=0).stop 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 shutdown_connections 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc8540a1c70 0x7fc8540a4130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc86c073f00 0x7fc86c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc86c106470 0x7fc86c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 --2- 172.21.15.107:0/2932434357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc86c1013a0 0x7fc86c073480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 >> 172.21.15.107:0/2932434357 conn(0x7fc86c0fcc00 msgr2=0x7fc86c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 shutdown_connections 2024-03-11T19:56:31.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/724: read d4/d7/d1f/d2b/d24/f5b [78183,24727] 0 2024-03-11T19:56:31.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/725: stat d4/d7/d1f/d2b/d19/d9a/fcd 0 2024-03-11T19:56:31.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/726: fsync d4/d7/d1f/f52 0 2024-03-11T19:56:31.674 INFO:tasks.workunit.client.0.smithi107.stdout:5/816: rmdir da/d14/d38 39 2024-03-11T19:56:31.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/727: dread - d4/d7/d1f/d2b/d19/d75/fe5 zero size 2024-03-11T19:56:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:31.671+0000 7fc873a00640 1 -- 172.21.15.107:0/2932434357 wait complete. 2024-03-11T19:56:31.675 INFO:tasks.workunit.client.0.smithi107.stdout:7/822: link d6/d9/fd9 d6/da/d16/d5c/d7d/d60/d61/d78/f102 0 2024-03-11T19:56:31.676 INFO:tasks.workunit.client.0.smithi107.stdout:8/621: rmdir dd/de/d1b/d20/d56 39 2024-03-11T19:56:31.677 INFO:tasks.workunit.client.0.smithi107.stdout:4/728: creat d4/d7/d1f/d2b/d19/d75/fe6 x:0 0 0 2024-03-11T19:56:31.678 INFO:tasks.workunit.client.0.smithi107.stdout:7/823: mkdir d6/da/d16/d91/dc8/d80/d103 0 2024-03-11T19:56:31.681 INFO:tasks.workunit.client.0.smithi107.stdout:4/729: link d4/d7/ca d4/d7/d2f/d3b/ce7 0 2024-03-11T19:56:31.682 INFO:tasks.workunit.client.0.smithi107.stdout:4/730: fsync d4/d7/d2f/d3c/f96 0 2024-03-11T19:56:31.682 INFO:tasks.workunit.client.0.smithi107.stdout:4/731: chown f3 98529926 1 2024-03-11T19:56:31.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/824: mkdir d6/da/d49/dee/d104 0 2024-03-11T19:56:31.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/732: dread d4/d7/d2f/d3c/f81 [0,4194304] 0 2024-03-11T19:56:31.696 INFO:tasks.workunit.client.0.smithi107.stdout:4/733: creat d4/d7/d2f/fe8 x:0 0 0 2024-03-11T19:56:31.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/870: write d1/d10/d21/d43/d48/d98/f9e [737907,51663] 0 2024-03-11T19:56:31.698 INFO:tasks.workunit.client.0.smithi107.stdout:4/734: mkdir d4/d7/d1f/d2b/d19/d75/d8e/de9 0 2024-03-11T19:56:31.698 INFO:tasks.workunit.client.0.smithi107.stdout:4/735: chown d4/d7/d2f 6509 1 2024-03-11T19:56:31.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/871: getdents d1/dbb/de3 0 2024-03-11T19:56:31.702 INFO:tasks.workunit.client.0.smithi107.stdout:1/742: rename d6/d7e/da5/le6 to d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3/lf2 0 2024-03-11T19:56:31.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/713: rename de to de/d11/d23/d32/d34/d39/d55/dd8 22 2024-03-11T19:56:31.702 INFO:tasks.workunit.client.0.smithi107.stdout:1/743: chown d6/d7e/f8b 97708334 1 2024-03-11T19:56:31.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/744: stat d6/d7e/da5/lee 0 2024-03-11T19:56:31.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/731: dread d4/d9/f86 [0,4194304] 0 2024-03-11T19:56:31.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/745: read - d6/d5b/f85 zero size 2024-03-11T19:56:31.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/746: write d6/d7e/d81/d27/d2c/d7d/dd8/ff1 [914148,82638] 0 2024-03-11T19:56:31.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/747: chown d6/d7e/da5/ldf 1173628417 1 2024-03-11T19:56:31.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/714: creat de/d11/d23/d24/da4/dae/dcd/fd9 x:0 0 0 2024-03-11T19:56:31.704 INFO:tasks.workunit.client.0.smithi107.stdout:9/732: write d4/d1d/d1f/d28/d2c/d96/f7a [2418171,70888] 0 2024-03-11T19:56:31.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/748: creat d6/d7e/d81/d27/d2c/d32/dcd/ff3 x:0 0 0 2024-03-11T19:56:31.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/749: readlink d6/d7e/d81/d27/lbe 0 2024-03-11T19:56:31.709 INFO:tasks.workunit.client.0.smithi107.stdout:0/715: mkdir de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda 0 2024-03-11T19:56:31.710 INFO:tasks.workunit.client.0.smithi107.stdout:0/716: write de/d11/d23/d32/d34/d39/d3a/d72/f98 [830903,24262] 0 2024-03-11T19:56:31.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/825: dwrite d6/da/d16/f98 [0,4194304] 0 2024-03-11T19:56:31.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/826: symlink d6/d21/dab/l105 0 2024-03-11T19:56:31.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/827: creat d6/da/d49/dee/d104/f106 x:0 0 0 2024-03-11T19:56:31.733 INFO:tasks.workunit.client.0.smithi107.stdout:5/817: dread da/f85 [0,4194304] 0 2024-03-11T19:56:31.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/818: unlink da/d14/d96/dc1/cdb 0 2024-03-11T19:56:31.740 INFO:tasks.workunit.client.0.smithi107.stdout:6/649: rename d10/d16/d2b/f9a to d10/d19/d3b/d70/fd8 0 2024-03-11T19:56:31.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/736: dread d4/d7/db/d5c/fc4 [0,4194304] 0 2024-03-11T19:56:31.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/650: rmdir d10/d16/d1f/d51/d76/d7f 39 2024-03-11T19:56:31.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/651: getdents d10/d19/d38 0 2024-03-11T19:56:31.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/679: rename d1/d1a/d34/d4f/f83 to d1/d1d/d64/fe3 0 2024-03-11T19:56:31.747 INFO:tasks.workunit.client.0.smithi107.stdout:2/680: mknod d1/ce4 0 2024-03-11T19:56:31.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/681: readlink d1/lc5 0 2024-03-11T19:56:31.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/682: fdatasync d1/d21/d35/d67/fc0 0 2024-03-11T19:56:31.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/683: write d1/d1a/d34/fb5 [2580920,53716] 0 2024-03-11T19:56:31.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/684: truncate d1/d1a/d1e/d55/fca 440803 0 2024-03-11T19:56:31.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/685: truncate d1/d1d/d64/fa1 29196 0 2024-03-11T19:56:31.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/737: rename d4/d7/f2c to d4/d7/db/db8/fea 0 2024-03-11T19:56:31.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/738: fdatasync d4/d7/da0/fe2 0 2024-03-11T19:56:31.750 INFO:tasks.workunit.client.0.smithi107.stdout:2/686: mknod d1/d1a/d34/d4f/d63/ce5 0 2024-03-11T19:56:31.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/739: creat d4/d7/d1f/d2b/d24/feb x:0 0 0 2024-03-11T19:56:31.766 INFO:tasks.workunit.client.0.smithi107.stdout:1/750: dread d6/f1d [0,4194304] 0 2024-03-11T19:56:31.770 INFO:tasks.workunit.client.0.smithi107.stdout:3/872: dwrite d1/d10/d21/d43/d48/fe9 [0,4194304] 0 2024-03-11T19:56:31.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/873: chown d1/d10/d26/c60 299123723 1 2024-03-11T19:56:31.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/874: fsync d1/d10/d13/d4a/d4c/d49/f54 0 2024-03-11T19:56:31.776 INFO:tasks.workunit.client.0.smithi107.stdout:8/622: dwrite dd/f58 [0,4194304] 0 2024-03-11T19:56:31.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/751: unlink d6/f54 0 2024-03-11T19:56:31.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/828: dread d6/da/d16/d5c/d7d/fa9 [0,4194304] 0 2024-03-11T19:56:31.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/829: chown d6/da/d16/f2c 12709325 1 2024-03-11T19:56:31.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/875: creat d1/d10/d13/d6c/d2d/d4d/d91/dcd/f116 x:0 0 0 2024-03-11T19:56:31.784 INFO:tasks.workunit.client.0.smithi107.stdout:2/687: dread d1/d21/d35/d67/f6e [0,4194304] 0 2024-03-11T19:56:31.784 INFO:tasks.workunit.client.0.smithi107.stdout:7/830: symlink d6/da/d16/d91/dc8/d80/d103/l107 0 2024-03-11T19:56:31.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/623: dread dd/de/d49/d7e/fae [0,4194304] 0 2024-03-11T19:56:31.792 INFO:tasks.workunit.client.0.smithi107.stdout:3/876: mkdir d1/d10/d21/d43/d48/d117 0 2024-03-11T19:56:31.792 INFO:tasks.workunit.client.0.smithi107.stdout:8/624: read dd/de/d24/d28/d47/fb1 [91527,109476] 0 2024-03-11T19:56:31.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/652: dread d10/d16/f1e [4194304,4194304] 0 2024-03-11T19:56:31.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/653: readlink d10/d16/d2b/d39/l49 0 2024-03-11T19:56:31.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/717: dread de/d11/d23/d24/d4f/f78 [0,4194304] 0 2024-03-11T19:56:31.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/718: write de/d58/f7f [338646,83584] 0 2024-03-11T19:56:31.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/719: fsync de/d11/d23/d24/d4f/f78 0 2024-03-11T19:56:31.795 INFO:tasks.workunit.client.0.smithi107.stdout:3/877: getdents d1/d10/d13/d4a/d4c 0 2024-03-11T19:56:31.795 INFO:tasks.workunit.client.0.smithi107.stdout:9/733: dwrite d4/d9/d4d/f8e [0,4194304] 0 2024-03-11T19:56:31.795 INFO:tasks.workunit.client.0.smithi107.stdout:0/720: chown de/d11/d74/da2/dd3/dd4/l94 23 1 2024-03-11T19:56:31.797 INFO:tasks.workunit.client.0.smithi107.stdout:8/625: creat dd/de/d1b/d20/d56/fb8 x:0 0 0 2024-03-11T19:56:31.800 INFO:tasks.workunit.client.0.smithi107.stdout:2/688: dread d1/d1a/d34/d4f/d63/f6f [0,4194304] 0 2024-03-11T19:56:31.800 INFO:tasks.workunit.client.0.smithi107.stdout:8/626: creat dd/dac/fb9 x:0 0 0 2024-03-11T19:56:31.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/878: unlink d1/d10/d2a/d7e/d8b/fc6 0 2024-03-11T19:56:31.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/879: chown d1/d10/d4e/d73/fc3 116544524 1 2024-03-11T19:56:31.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/627: getdents dd/de/d24/d28 0 2024-03-11T19:56:31.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/628: write dd/d13/d32/d51/f85 [667813,1520] 0 2024-03-11T19:56:31.802 INFO:tasks.workunit.client.0.smithi107.stdout:0/721: symlink de/d11/ldb 0 2024-03-11T19:56:31.803 INFO:tasks.workunit.client.0.smithi107.stdout:0/722: getdents de/d11/d23/d24/da4 0 2024-03-11T19:56:31.805 INFO:tasks.workunit.client.0.smithi107.stdout:0/723: unlink de/l68 0 2024-03-11T19:56:31.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/689: write d1/d38/d75/fa9 [195137,81725] 0 2024-03-11T19:56:31.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/724: chown de/d11/d74/da2/dd3/dd4/l9a 1 1 2024-03-11T19:56:31.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/725: chown de/d11/d23/d32/d34/d39/d55/d7c/c9c 6636 1 2024-03-11T19:56:31.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/726: chown de/d11/d23/d32/d34/d39/d3a/c6b 61546 1 2024-03-11T19:56:31.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/690: link d1/d21/c4c d1/d38/d75/ce6 0 2024-03-11T19:56:31.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/691: stat d1/d21/dbb/fd0 0 2024-03-11T19:56:31.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/727: write de/d11/d74/da2/dd3/dd4/dbf/dd1/fd6 [423766,115961] 0 2024-03-11T19:56:31.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/692: rmdir d1/d1d/d64/d97 39 2024-03-11T19:56:31.810 INFO:tasks.workunit.client.0.smithi107.stdout:0/728: symlink de/d11/d23/d24/da4/dae/ldc 0 2024-03-11T19:56:31.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/819: dwrite da/d14/d43/d9e/dad/fbb [0,4194304] 0 2024-03-11T19:56:31.814 INFO:tasks.workunit.client.0.smithi107.stdout:0/729: symlink de/d11/d23/d32/d34/d39/ldd 0 2024-03-11T19:56:31.816 INFO:tasks.workunit.client.0.smithi107.stdout:5/820: rmdir da/d14/d18/db2 39 2024-03-11T19:56:31.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/730: mknod de/d11/d23/d24/d83/dc4/dd2/cde 0 2024-03-11T19:56:31.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/731: read de/d11/d23/d24/f69 [1564322,118272] 0 2024-03-11T19:56:31.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/629: dread dd/d16/f33 [0,4194304] 0 2024-03-11T19:56:31.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/630: rename dd/d16/l72 to dd/dac/lba 0 2024-03-11T19:56:31.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/752: dwrite d6/d7e/d81/d27/f7a [0,4194304] 0 2024-03-11T19:56:31.841 INFO:tasks.workunit.client.0.smithi107.stdout:7/831: dwrite d6/d9/dae/fb8 [0,4194304] 0 2024-03-11T19:56:31.844 INFO:tasks.workunit.client.0.smithi107.stdout:6/654: dread d10/d16/d24/d2e/f36 [0,4194304] 0 2024-03-11T19:56:31.845 INFO:tasks.workunit.client.0.smithi107.stdout:6/655: creat d10/d16/d2b/d39/dca/fd9 x:0 0 0 2024-03-11T19:56:31.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/631: dread dd/d13/d32/d51/fb3 [0,4194304] 0 2024-03-11T19:56:31.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/632: mkdir dd/de/d24/d28/d47/dbb 0 2024-03-11T19:56:31.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/633: chown dd/de/d1b/c77 636411 1 2024-03-11T19:56:31.861 INFO:tasks.workunit.client.0.smithi107.stdout:8/634: symlink dd/dac/lbc 0 2024-03-11T19:56:31.862 INFO:tasks.workunit.client.0.smithi107.stdout:8/635: mknod dd/de/d1b/d4b/cbd 0 2024-03-11T19:56:31.871 INFO:tasks.workunit.client.0.smithi107.stdout:9/734: dwrite d4/d9/d4d/f8e [0,4194304] 0 2024-03-11T19:56:31.871 INFO:tasks.workunit.client.0.smithi107.stdout:9/735: stat d4/d1d/d1f/d78/f90 0 2024-03-11T19:56:31.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/736: symlink d4/d1d/d1f/l103 0 2024-03-11T19:56:31.885 INFO:tasks.workunit.client.0.smithi107.stdout:8/636: dread dd/d13/d18/d81/f94 [0,4194304] 0 2024-03-11T19:56:31.885 INFO:tasks.workunit.client.0.smithi107.stdout:8/637: getdents dd/de/d24/d28 0 2024-03-11T19:56:32.027 INFO:tasks.workunit.client.0.smithi107.stdout:6/656: dwrite d10/d16/d1f/d51/d76/f93 [0,4194304] 0 2024-03-11T19:56:32.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/693: dwrite d1/d21/d35/f54 [4194304,4194304] 0 2024-03-11T19:56:32.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/694: mknod d1/d1a/d34/d4f/d63/daa/ce7 0 2024-03-11T19:56:32.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/732: link de/l1c de/d11/d23/d32/d34/d39/d55/d7c/ldf 0 2024-03-11T19:56:32.044 INFO:tasks.workunit.client.0.smithi107.stdout:8/638: rename dd/de/d49/d46/fad to dd/d13/d32/fbe 0 2024-03-11T19:56:32.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/733: rmdir de/d11/d23/d32/d34/d39/d55/d7c/dd7 39 2024-03-11T19:56:32.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/639: rename dd/de/d24/d28/d47/c62 to dd/de/d24/d28/d47/dbb/cbf 0 2024-03-11T19:56:32.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/640: stat dd/de/d1b/d4b/fb7 0 2024-03-11T19:56:32.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/641: chown dd/de/d1b/f9a 0 1 2024-03-11T19:56:32.055 INFO:tasks.workunit.client.0.smithi107.stdout:8/642: mknod dd/de/d1b/cc0 0 2024-03-11T19:56:32.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/753: write d6/d7e/d81/d27/d2c/d32/d58/fc8 [2020268,16447] 0 2024-03-11T19:56:32.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/754: write d6/d7e/d81/f24 [1049555,29567] 0 2024-03-11T19:56:32.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/657: read d10/d16/d2b/da5/faa [3410322,3285] 0 2024-03-11T19:56:32.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/658: stat d10/d16/d24/d2e/d9d/fa8 0 2024-03-11T19:56:32.075 INFO:tasks.workunit.client.0.smithi107.stdout:6/659: creat d10/d16/d1f/d51/d76/d7f/dbd/fda x:0 0 0 2024-03-11T19:56:32.078 INFO:tasks.workunit.client.0.smithi107.stdout:0/734: dread de/f40 [0,4194304] 0 2024-03-11T19:56:32.078 INFO:tasks.workunit.client.0.smithi107.stdout:6/660: mkdir d10/d19/d3b/ddb 0 2024-03-11T19:56:32.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/735: creat de/d9e/dc8/fe0 x:0 0 0 2024-03-11T19:56:32.081 INFO:tasks.workunit.client.0.smithi107.stdout:6/661: readlink d10/d19/d38/d60/db1/l4f 0 2024-03-11T19:56:32.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/695: dread d1/d1a/d34/d4f/d63/f6f [0,4194304] 0 2024-03-11T19:56:32.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/696: write d1/d21/d35/d67/fc0 [560198,127797] 0 2024-03-11T19:56:32.092 INFO:tasks.workunit.client.0.smithi107.stdout:1/755: dread d6/d5b/f78 [0,4194304] 0 2024-03-11T19:56:32.093 INFO:tasks.workunit.client.0.smithi107.stdout:0/736: write de/d11/d23/d32/d34/d39/d3a/f66 [323861,27350] 0 2024-03-11T19:56:32.096 INFO:tasks.workunit.client.0.smithi107.stdout:0/737: creat de/d11/d74/da2/fe1 x:0 0 0 2024-03-11T19:56:32.097 INFO:tasks.workunit.client.0.smithi107.stdout:5/821: dwrite da/d14/d43/d7f/fc2 [0,4194304] 0 2024-03-11T19:56:32.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/756: rmdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3 39 2024-03-11T19:56:32.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/662: dread d10/d16/d2b/da5/faa [0,4194304] 0 2024-03-11T19:56:32.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/757: truncate d6/d7e/d81/d27/d2c/d32/dc1/f96 54617 0 2024-03-11T19:56:32.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/663: truncate d10/d19/d3b/d70/dbb/fd6 14554 0 2024-03-11T19:56:32.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/664: chown c7 182950154 1 2024-03-11T19:56:32.115 INFO:tasks.workunit.client.0.smithi107.stdout:6/665: rename d10/d16/d1f/d51/c9b to d10/d16/d1f/d51/cdc 0 2024-03-11T19:56:32.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/697: dread d1/d1a/d34/fb5 [0,4194304] 0 2024-03-11T19:56:32.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/698: fsync d1/d1d/d64/fa1 0 2024-03-11T19:56:32.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/699: creat d1/d21/d35/d67/fe8 x:0 0 0 2024-03-11T19:56:32.140 INFO:tasks.workunit.client.0.smithi107.stdout:5/822: dwrite da/d14/d18/d1e/d9b/d73/fa3 [0,4194304] 0 2024-03-11T19:56:32.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/758: dwrite d6/d7e/d81/d27/f47 [0,4194304] 0 2024-03-11T19:56:32.164 INFO:tasks.workunit.client.0.smithi107.stdout:1/759: mkdir d6/d7e/d81/d27/d2c/d32/dc1/d95/dbb/df4 0 2024-03-11T19:56:32.171 INFO:tasks.workunit.client.0.smithi107.stdout:5/823: dread da/d31/d42/f11 [0,4194304] 0 2024-03-11T19:56:32.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/824: mkdir da/d14/d18/dff 0 2024-03-11T19:56:32.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/825: dread da/d14/d18/f1c [0,4194304] 0 2024-03-11T19:56:32.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/826: link da/d14/d18/d1e/d9b/d73/cbf da/d14/d38/d59/de5/c100 0 2024-03-11T19:56:32.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/827: rmdir da/d14/d38/d59/de5 39 2024-03-11T19:56:32.232 INFO:tasks.workunit.client.0.smithi107.stdout:6/666: rename d10/d19/f61 to d10/d16/d2b/d8d/fdd 0 2024-03-11T19:56:32.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/700: rename d1/d1d/d80/fd3 to d1/d21/dda/fe9 0 2024-03-11T19:56:32.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/701: chown d1/d1d/d80 2669 1 2024-03-11T19:56:32.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/702: mknod d1/d21/dda/cea 0 2024-03-11T19:56:32.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/703: read - d1/f7b zero size 2024-03-11T19:56:32.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/704: dread - d1/d1a/d34/d4f/db3/fb7 zero size 2024-03-11T19:56:32.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/705: readlink d1/d1a/d34/d4f/lb1 0 2024-03-11T19:56:32.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/706: readlink d1/d1a/d34/le1 0 2024-03-11T19:56:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/707: symlink d1/d1a/d34/d4f/db3/leb 0 2024-03-11T19:56:32.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/708: creat d1/d1a/d8e/fec x:0 0 0 2024-03-11T19:56:32.250 INFO:tasks.workunit.client.0.smithi107.stdout:2/709: mknod d1/d1a/d8e/ced 0 2024-03-11T19:56:32.259 INFO:tasks.workunit.client.0.smithi107.stdout:6/667: link d10/d19/d3b/f5e d10/d16/fde 0 2024-03-11T19:56:32.265 INFO:tasks.workunit.client.0.smithi107.stdout:6/668: rmdir d10/d16/d2b/d39/d6a 39 2024-03-11T19:56:32.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/760: rename d6/f14 to d6/d5b/ff5 0 2024-03-11T19:56:32.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/761: readlink d6/d7e/da5/lbf 0 2024-03-11T19:56:32.273 INFO:tasks.workunit.client.0.smithi107.stdout:5/828: rename c1 to da/d14/d96/dc1/c101 0 2024-03-11T19:56:32.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/762: link d6/d7e/d81/l31 d6/d7e/d81/lf6 0 2024-03-11T19:56:32.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/763: fdatasync d6/d19/f43 0 2024-03-11T19:56:32.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/764: getdents d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82 0 2024-03-11T19:56:32.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/765: getdents d6/d7e/d81/d27/d2c/d32/dcd 0 2024-03-11T19:56:32.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/766: stat d6/d7e/d81/d27/f47 0 2024-03-11T19:56:32.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/767: chown d6/d7e 8461065 1 2024-03-11T19:56:32.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/669: dread d10/d16/fab [0,4194304] 0 2024-03-11T19:56:32.295 INFO:tasks.workunit.client.0.smithi107.stdout:6/670: creat d10/d16/d1f/d51/d76/fdf x:0 0 0 2024-03-11T19:56:32.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/671: creat d10/d19/fe0 x:0 0 0 2024-03-11T19:56:32.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/672: creat d10/d16/d1f/d51/d7a/fe1 x:0 0 0 2024-03-11T19:56:32.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/673: mkdir d10/d16/d2b/de2 0 2024-03-11T19:56:32.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/674: read d10/d16/d2b/d39/f6d [203788,80111] 0 2024-03-11T19:56:32.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/675: chown d10/d19/d3b/d70/dbb/fd6 2817951 1 2024-03-11T19:56:32.318 INFO:tasks.workunit.client.0.smithi107.stdout:5/829: dread da/d31/f75 [0,4194304] 0 2024-03-11T19:56:32.353 INFO:tasks.workunit.client.0.smithi107.stdout:5/830: dread da/d14/d43/d47/fa2 [0,4194304] 0 2024-03-11T19:56:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/643: sync 2024-03-11T19:56:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/737: sync 2024-03-11T19:56:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/880: sync 2024-03-11T19:56:32.367 INFO:tasks.workunit.client.0.smithi107.stdout:7/832: sync 2024-03-11T19:56:32.367 INFO:tasks.workunit.client.0.smithi107.stdout:4/740: sync 2024-03-11T19:56:32.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/833: readlink d6/da/d16/d5c/d7d/d60/lb6 0 2024-03-11T19:56:32.368 INFO:tasks.workunit.client.0.smithi107.stdout:4/741: fsync f0 0 2024-03-11T19:56:32.374 INFO:tasks.workunit.client.0.smithi107.stdout:8/644: unlink dd/d13/d32/f7b 0 2024-03-11T19:56:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:3/881: mknod d1/d10/d13/d4a/d4c/d49/c118 0 2024-03-11T19:56:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:3/882: rename d1 to d1/d10/d4e/d73/d8d/d119 22 2024-03-11T19:56:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:4/742: creat d4/d7/d1f/d2b/d93/dc3/dd0/fec x:0 0 0 2024-03-11T19:56:32.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/883: symlink d1/d10/d2a/d7e/d8b/l11a 0 2024-03-11T19:56:32.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/884: readlink d1/dbb/de3/dfd/leb 0 2024-03-11T19:56:32.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/676: dwrite d10/d3d/d54/fa2 [0,4194304] 0 2024-03-11T19:56:32.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/677: stat d10/d16/d2b/d39 0 2024-03-11T19:56:32.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/678: symlink d10/d19/d3b/d70/le3 0 2024-03-11T19:56:32.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/743: dread d4/d7/d1f/d2b/d19/d75/d8e/da5/fa6 [0,4194304] 0 2024-03-11T19:56:32.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/744: fsync d4/d7/da0/fb4 0 2024-03-11T19:56:32.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/745: creat d4/d7/db/d5e/fed x:0 0 0 2024-03-11T19:56:32.394 INFO:tasks.workunit.client.0.smithi107.stdout:4/746: creat d4/d7/d2f/d33/fee x:0 0 0 2024-03-11T19:56:32.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/747: symlink d4/d7/db/d5c/lef 0 2024-03-11T19:56:32.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/748: creat d4/d7/d1f/d2b/d19/d75/d8e/ff0 x:0 0 0 2024-03-11T19:56:32.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/749: chown d4/d7/d1f/d2b/d93/dc3/dd0/d79/dbb 16279 1 2024-03-11T19:56:32.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/831: read da/d14/d18/d1e/d9b/fa4 [4647,57301] 0 2024-03-11T19:56:32.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/832: truncate da/d14/d96/dc1/fdc 1476364 0 2024-03-11T19:56:32.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/833: creat da/d14/d38/d59/de5/f102 x:0 0 0 2024-03-11T19:56:32.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/834: dread - da/d14/d38/d59/d5f/dc9/fb8 zero size 2024-03-11T19:56:32.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/835: creat da/d14/d96/dc1/f103 x:0 0 0 2024-03-11T19:56:32.418 INFO:tasks.workunit.client.0.smithi107.stdout:5/836: creat da/d14/d38/d46/dd9/f104 x:0 0 0 2024-03-11T19:56:32.429 INFO:tasks.workunit.client.0.smithi107.stdout:5/837: dread da/d14/d38/d59/d5f/f80 [0,4194304] 0 2024-03-11T19:56:32.431 INFO:tasks.workunit.client.0.smithi107.stdout:5/838: mkdir da/d14/d18/dff/d105 0 2024-03-11T19:56:32.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/885: dwrite d1/d10/d21/d43/d48/f56 [0,4194304] 0 2024-03-11T19:56:32.435 INFO:tasks.workunit.client.0.smithi107.stdout:3/886: fdatasync d1/dbb/fe4 0 2024-03-11T19:56:32.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/645: link dd/de/d1b/d4b/fb7 dd/d13/d18/fc1 0 2024-03-11T19:56:32.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/646: dread - dd/de/d1b/f83 zero size 2024-03-11T19:56:32.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/647: creat dd/de/d1b/d2c/fc2 x:0 0 0 2024-03-11T19:56:32.441 INFO:tasks.workunit.client.0.smithi107.stdout:8/648: mknod dd/de/cc3 0 2024-03-11T19:56:32.441 INFO:tasks.workunit.client.0.smithi107.stdout:8/649: mknod dd/de/d49/d46/cc4 0 2024-03-11T19:56:32.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/679: dwrite d10/d16/d2b/d39/d6a/fd1 [0,4194304] 0 2024-03-11T19:56:32.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/680: symlink d10/d19/d3b/d70/d8a/le4 0 2024-03-11T19:56:32.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/650: unlink dd/de/d24/l55 0 2024-03-11T19:56:32.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/681: truncate d10/d3d/d54/fa2 2523839 0 2024-03-11T19:56:32.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/682: getdents d10/d3d/d54 0 2024-03-11T19:56:32.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/683: readlink d10/d19/d3b/d70/le3 0 2024-03-11T19:56:32.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/887: dread d1/d10/d13/d4a/d4c/ff3 [0,4194304] 0 2024-03-11T19:56:32.461 INFO:tasks.workunit.client.0.smithi107.stdout:3/888: truncate d1/d10/d21/d43/d48/fe9 510233 0 2024-03-11T19:56:32.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/834: dwrite d6/d9/d57/d65/fef [0,4194304] 0 2024-03-11T19:56:32.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/839: dread da/d31/f4f [0,4194304] 0 2024-03-11T19:56:32.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/840: fsync da/d14/d43/d47/fc7 0 2024-03-11T19:56:32.465 INFO:tasks.workunit.client.0.smithi107.stdout:3/889: creat d1/d10/d13/d6c/d2d/d4d/d91/f11b x:0 0 0 2024-03-11T19:56:32.481 INFO:tasks.workunit.client.0.smithi107.stdout:9/738: dwrite d4/d9/dae/ffa [0,4194304] 0 2024-03-11T19:56:32.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/739: mkdir d4/d1d/d5c/d64/d104 0 2024-03-11T19:56:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/740: stat d4/d9/fa 0 2024-03-11T19:56:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/741: dread - d4/d1d/d1f/d28/fc1 zero size 2024-03-11T19:56:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/742: fsync d4/d1c/d37/ff3 0 2024-03-11T19:56:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/743: chown d4/d9/f101 181718434 1 2024-03-11T19:56:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:3/890: dread d1/f5 [4194304,4194304] 0 2024-03-11T19:56:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/651: dwrite dd/de/d1b/d20/d5e/fa9 [0,4194304] 0 2024-03-11T19:56:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/744: unlink d4/d9/ceb 0 2024-03-11T19:56:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/891: truncate d1/d10/d13/f2c 5347420 0 2024-03-11T19:56:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/892: getdents d1/d10/d21/d43/d48/d117 0 2024-03-11T19:56:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/893: write d1/d10/d26/d17/fae [977540,105852] 0 2024-03-11T19:56:32.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/652: symlink dd/de/d24/lc5 0 2024-03-11T19:56:32.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/653: truncate dd/fb6 938153 0 2024-03-11T19:56:32.498 INFO:tasks.workunit.client.0.smithi107.stdout:9/745: rmdir d4/d1d/d1f/d28/d2c 39 2024-03-11T19:56:32.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/654: symlink dd/dac/lc6 0 2024-03-11T19:56:32.501 INFO:tasks.workunit.client.0.smithi107.stdout:9/746: fsync d4/d1d/d1f/d28/d2c/d96/dd3/fdf 0 2024-03-11T19:56:32.501 INFO:tasks.workunit.client.0.smithi107.stdout:9/747: dread - d4/d9/dae/d6d/f68 zero size 2024-03-11T19:56:32.501 INFO:tasks.workunit.client.0.smithi107.stdout:9/748: chown d4/d9/dae/d6d/f68 255842923 1 2024-03-11T19:56:32.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/749: mkdir d4/d1d/d1f/d78/d105 0 2024-03-11T19:56:32.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/750: dread - d4/d1c/d37/f75 zero size 2024-03-11T19:56:32.505 INFO:tasks.workunit.client.0.smithi107.stdout:9/751: mkdir d4/d106 0 2024-03-11T19:56:32.506 INFO:tasks.workunit.client.0.smithi107.stdout:3/894: write d1/d10/d4e/d73/d8d/f95 [5225094,75551] 0 2024-03-11T19:56:32.506 INFO:tasks.workunit.client.0.smithi107.stdout:3/895: stat d1/d10/d13/d4a/d4c/c67 0 2024-03-11T19:56:32.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/752: write d4/d9/dae/f77 [3628624,63386] 0 2024-03-11T19:56:32.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/753: mknod d4/d1d/d1f/c107 0 2024-03-11T19:56:32.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/754: chown d4/d1d/d5c/d97/fbd 361026 1 2024-03-11T19:56:32.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/755: fsync d4/d1d/d5c/d97/fe2 0 2024-03-11T19:56:32.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/756: creat d4/d1d/d5c/d97/f108 x:0 0 0 2024-03-11T19:56:32.528 INFO:tasks.workunit.client.0.smithi107.stdout:9/757: symlink d4/d1d/d1f/d28/d2c/l109 0 2024-03-11T19:56:32.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/841: dread da/f86 [0,4194304] 0 2024-03-11T19:56:32.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/842: creat da/d14/d18/f106 x:0 0 0 2024-03-11T19:56:32.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/758: rename d4/d1d/d5c/cac to d4/d1c/d37/dd9/c10a 0 2024-03-11T19:56:32.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/843: mknod da/d14/d18/d44/c107 0 2024-03-11T19:56:32.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/844: write da/d14/d96/dc1/f103 [340063,29742] 0 2024-03-11T19:56:32.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/759: rename d4/d1d/d1f/cab to d4/d1d/d4b/c10b 0 2024-03-11T19:56:32.541 INFO:tasks.workunit.client.0.smithi107.stdout:5/845: truncate da/d14/d38/d59/dbd/fde 2067643 0 2024-03-11T19:56:32.541 INFO:tasks.workunit.client.0.smithi107.stdout:5/846: read - da/d14/d38/dd0/ff9 zero size 2024-03-11T19:56:32.541 INFO:tasks.workunit.client.0.smithi107.stdout:5/847: creat da/d14/d18/f108 x:0 0 0 2024-03-11T19:56:32.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/684: dwrite d10/d19/d3b/f99 [0,4194304] 0 2024-03-11T19:56:32.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/685: chown d10/d16/d2b/d39/d6a/d6e/la0 270377761 1 2024-03-11T19:56:32.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/686: mknod d10/d3d/ce5 0 2024-03-11T19:56:32.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/760: mknod d4/d1c/d47/d8a/c10c 0 2024-03-11T19:56:32.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/761: truncate d4/fcb 908962 0 2024-03-11T19:56:32.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/655: dwrite dd/de/d1b/d20/d56/d87/f8f [0,4194304] 0 2024-03-11T19:56:32.545 INFO:tasks.workunit.client.0.smithi107.stdout:3/896: dread d1/d10/d26/d17/fae [0,4194304] 0 2024-03-11T19:56:32.545 INFO:tasks.workunit.client.0.smithi107.stdout:3/897: write d1/d45/f75 [2680677,46693] 0 2024-03-11T19:56:32.545 INFO:tasks.workunit.client.0.smithi107.stdout:6/687: truncate d10/d16/d2b/da5/faa 4107979 0 2024-03-11T19:56:32.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/848: symlink da/d31/l109 0 2024-03-11T19:56:32.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/849: chown da/d14/d18/f95 6202 1 2024-03-11T19:56:32.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/850: mkdir da/d14/d18/d44/dcd/d10a 0 2024-03-11T19:56:32.548 INFO:tasks.workunit.client.0.smithi107.stdout:0/738: sync 2024-03-11T19:56:32.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/762: mknod d4/d1d/d1f/d28/d2c/d96/dd3/c10d 0 2024-03-11T19:56:32.552 INFO:tasks.workunit.client.0.smithi107.stdout:8/656: creat dd/de/d1b/d20/d5e/fc7 x:0 0 0 2024-03-11T19:56:32.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/835: dwrite d6/da/d16/d5c/d7d/f5e [0,4194304] 0 2024-03-11T19:56:32.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/851: mkdir da/d14/d43/d9e/dad/d10b 0 2024-03-11T19:56:32.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/739: dread - de/d11/d74/da2/dd3/dd4/f48 zero size 2024-03-11T19:56:32.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/852: rmdir da/d14/df7 0 2024-03-11T19:56:32.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/853: chown da/d14/d96/f88 3946073 1 2024-03-11T19:56:32.557 INFO:tasks.workunit.client.0.smithi107.stdout:8/657: mkdir dd/de/d1b/d20/d5e/dc8 0 2024-03-11T19:56:32.557 INFO:tasks.workunit.client.0.smithi107.stdout:8/658: chown dd/d13/d18/d81/f8d 66 1 2024-03-11T19:56:32.559 INFO:tasks.workunit.client.0.smithi107.stdout:8/659: link dd/d13/d32/d51/fb3 dd/d13/fc9 0 2024-03-11T19:56:32.561 INFO:tasks.workunit.client.0.smithi107.stdout:8/660: truncate dd/de/d49/f42 1068251 0 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:2/710: sync 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:1/768: sync 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:1/769: fsync d6/d5b/f78 0 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:2/711: write d1/d1a/d34/d4f/db3/fb7 [828165,122512] 0 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:1/770: write d6/d7e/d81/d27/f38 [4532713,96215] 0 2024-03-11T19:56:32.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/750: sync 2024-03-11T19:56:32.571 INFO:tasks.workunit.client.0.smithi107.stdout:2/712: creat d1/d1a/d1e/d55/fee x:0 0 0 2024-03-11T19:56:32.571 INFO:tasks.workunit.client.0.smithi107.stdout:2/713: stat d1/d38/fc8 0 2024-03-11T19:56:32.572 INFO:tasks.workunit.client.0.smithi107.stdout:2/714: write d1/d38/d75/f95 [4592787,118523] 0 2024-03-11T19:56:32.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/836: dread d6/da/d16/fa5 [0,4194304] 0 2024-03-11T19:56:32.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/837: dread - d6/d9/d4e/ff7 zero size 2024-03-11T19:56:32.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/838: dread - d6/d9/d57/da2/ff5 zero size 2024-03-11T19:56:32.575 INFO:tasks.workunit.client.0.smithi107.stdout:8/661: write dd/d13/d32/fbe [1850429,21252] 0 2024-03-11T19:56:32.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/751: write d4/fc9 [41245,80493] 0 2024-03-11T19:56:32.582 INFO:tasks.workunit.client.0.smithi107.stdout:0/740: rename de/d11/d23/d24/lb5 to de/d11/d23/d24/da4/dae/le2 0 2024-03-11T19:56:32.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/752: dread d4/d7/d2f/d3c/f73 [0,4194304] 0 2024-03-11T19:56:32.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/688: dwrite d10/d16/d2b/f91 [0,4194304] 0 2024-03-11T19:56:32.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/689: chown d10/d19 4418 1 2024-03-11T19:56:32.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/741: dread de/d11/d23/d32/d34/d39/d3a/f66 [0,4194304] 0 2024-03-11T19:56:32.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/742: chown de/d11/d23/d32/d34/d39/d55/d7c/c9c 0 1 2024-03-11T19:56:32.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/743: chown de/d11/d23/d24/da4/dae/fd0 3456674 1 2024-03-11T19:56:32.597 INFO:tasks.workunit.client.0.smithi107.stdout:4/753: getdents d4/d7/d1f/d2b/d24/d26/d29 0 2024-03-11T19:56:32.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/744: dread de/d11/d23/f76 [0,4194304] 0 2024-03-11T19:56:32.618 INFO:tasks.workunit.client.0.smithi107.stdout:0/745: mknod de/d11/d23/d24/d83/dc4/ce3 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/715: dwrite d1/d1a/f2f [0,4194304] 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/716: fsync d1/d1a/d1e/d26/fb2 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/717: readlink d1/d1a/d34/d4f/d63/laf 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/718: write d1/d1d/d64/fe3 [671503,4884] 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/719: write d1/f10 [1634833,22586] 0 2024-03-11T19:56:32.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/720: write d1/d1d/d64/d97/fab [283940,95673] 0 2024-03-11T19:56:32.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/898: dwrite d1/d10/d21/d43/d48/f8e [4194304,4194304] 0 2024-03-11T19:56:32.635 INFO:tasks.workunit.client.0.smithi107.stdout:3/899: chown d1/d10/d13/d4a/d4c/ce0 104948861 1 2024-03-11T19:56:32.635 INFO:tasks.workunit.client.0.smithi107.stdout:2/721: link d1/cdc d1/d1d/cef 0 2024-03-11T19:56:32.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/900: creat d1/d10/d4e/d73/f11c x:0 0 0 2024-03-11T19:56:32.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/690: dwrite d10/d16/d1f/d51/d9c/fb0 [0,4194304] 0 2024-03-11T19:56:32.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/691: fsync d10/d16/d2b/d39/fcf 0 2024-03-11T19:56:32.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/692: stat d10/d16/d2b/d39 0 2024-03-11T19:56:32.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/693: chown d10/d16/d2b/l47 3 1 2024-03-11T19:56:32.650 INFO:tasks.workunit.client.0.smithi107.stdout:6/694: link d10/d16/d1f/f2a d10/d16/d1f/d51/db7/fe6 0 2024-03-11T19:56:32.652 INFO:tasks.workunit.client.0.smithi107.stdout:0/746: dwrite de/d58/f8b [0,4194304] 0 2024-03-11T19:56:32.652 INFO:tasks.workunit.client.0.smithi107.stdout:6/695: chown d10/d16/d1f/d51/d7a/f94 947532060 1 2024-03-11T19:56:32.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/747: mkdir de/d11/d23/d24/d83/dc4/de4 0 2024-03-11T19:56:32.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/748: readlink de/d11/d23/d32/d34/d39/d3a/lc3 0 2024-03-11T19:56:32.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/749: link de/d11/d23/d32/d34/d39/d3a/f5e de/d11/d23/d24/d8d/fe5 0 2024-03-11T19:56:32.692 INFO:tasks.workunit.client.0.smithi107.stdout:6/696: dwrite f3 [4194304,4194304] 0 2024-03-11T19:56:32.692 INFO:tasks.workunit.client.0.smithi107.stdout:6/697: fdatasync d10/d19/d3b/d70/dbb/fd6 0 2024-03-11T19:56:32.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/698: creat d10/d19/d3b/d70/dbb/fe7 x:0 0 0 2024-03-11T19:56:32.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/699: fsync d10/d16/d2b/d39/d6a/fd1 0 2024-03-11T19:56:32.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/700: dread - d10/d16/d2b/d39/f66 zero size 2024-03-11T19:56:32.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/701: dread d10/d19/d3b/d70/dbb/fd6 [0,4194304] 0 2024-03-11T19:56:32.703 INFO:tasks.workunit.client.0.smithi107.stdout:6/702: mknod d10/ce8 0 2024-03-11T19:56:32.703 INFO:tasks.workunit.client.0.smithi107.stdout:6/703: readlink d10/d16/d24/l2c 0 2024-03-11T19:56:32.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/704: dwrite d10/d19/f23 [0,4194304] 0 2024-03-11T19:56:32.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/705: chown d10/d19/d38/ld5 18 1 2024-03-11T19:56:32.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/706: dread d10/d16/d2b/d8d/fdd [0,4194304] 0 2024-03-11T19:56:32.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/707: dwrite d10/d16/d2b/da5/faa [0,4194304] 0 2024-03-11T19:56:32.815 INFO:tasks.workunit.client.0.smithi107.stdout:6/708: read d10/d16/d2b/d39/d6a/f83 [4707291,45525] 0 2024-03-11T19:56:32.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/709: write d10/fac [226924,100852] 0 2024-03-11T19:56:32.828 INFO:tasks.workunit.client.0.smithi107.stdout:6/710: symlink d10/d19/d38/d60/le9 0 2024-03-11T19:56:32.829 INFO:tasks.workunit.client.0.smithi107.stdout:6/711: creat d10/d16/d2b/d39/dca/fea x:0 0 0 2024-03-11T19:56:32.829 INFO:tasks.workunit.client.0.smithi107.stdout:6/712: readlink d10/d19/d38/ld5 0 2024-03-11T19:56:32.829 INFO:tasks.workunit.client.0.smithi107.stdout:6/713: write d10/d16/d1f/d51/d7a/fe1 [774779,12600] 0 2024-03-11T19:56:32.903 INFO:tasks.workunit.client.0.smithi107.stdout:6/714: dwrite d10/d16/d1f/d51/d76/fdf [0,4194304] 0 2024-03-11T19:56:32.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/715: mkdir d10/d16/d2b/deb 0 2024-03-11T19:56:33.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:32 smithi122 ceph-mon[36397]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 79 op/s 2024-03-11T19:56:33.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:32 smithi145 ceph-mon[36730]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 79 op/s 2024-03-11T19:56:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:32 smithi107 ceph-mon[31712]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 29 MiB/s wr, 79 op/s 2024-03-11T19:56:33.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/854: rename da/d14/d18/d1e/d9b/d72/fed to da/d14/d18/d44/f10c 0 2024-03-11T19:56:33.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/855: mknod da/d14/d18/d44/dcd/c10d 0 2024-03-11T19:56:33.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/856: write da/d14/d18/d1e/d9b/d73/ff2 [443398,63488] 0 2024-03-11T19:56:33.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/857: chown da/f85 6943 1 2024-03-11T19:56:33.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/858: write da/d14/d38/fe9 [538678,61527] 0 2024-03-11T19:56:33.242 INFO:tasks.workunit.client.0.smithi107.stdout:5/859: getdents da/d14/d18/d6d 0 2024-03-11T19:56:33.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/860: mknod da/d14/d43/d9e/dad/d10b/c10e 0 2024-03-11T19:56:33.244 INFO:tasks.workunit.client.0.smithi107.stdout:5/861: mkdir da/d14/d18/dff/d105/d10f 0 2024-03-11T19:56:33.244 INFO:tasks.workunit.client.0.smithi107.stdout:8/662: rename dd/de/d49/d46/d88 to dd/de/d1b/d20/d56/da4/dca 0 2024-03-11T19:56:33.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/663: creat dd/de/d49/d7e/fcb x:0 0 0 2024-03-11T19:56:33.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/754: rename d4/d7/db/d5e/l8b to d4/d7/d1f/d2b/d93/dc3/lf1 0 2024-03-11T19:56:33.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/755: chown d4/d7/d1f/d2b/d19/d75/d8e/da5/fa6 1066845573 1 2024-03-11T19:56:33.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/756: fdatasync f3 0 2024-03-11T19:56:33.250 INFO:tasks.workunit.client.0.smithi107.stdout:2/722: rename d1/f36 to d1/d1d/d64/d97/ff0 0 2024-03-11T19:56:33.253 INFO:tasks.workunit.client.0.smithi107.stdout:2/723: creat d1/d21/ff1 x:0 0 0 2024-03-11T19:56:33.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/901: rename d1/dbb/de3/dfd/leb to d1/d10/d21/d43/d48/d117/l11d 0 2024-03-11T19:56:33.255 INFO:tasks.workunit.client.0.smithi107.stdout:4/757: rmdir d4/d7/d2f/d33 39 2024-03-11T19:56:33.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/902: mknod d1/d10/d21/d43/d48/d117/c11e 0 2024-03-11T19:56:33.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/664: mknod dd/de/d24/d28/d47/dbb/ccc 0 2024-03-11T19:56:33.259 INFO:tasks.workunit.client.0.smithi107.stdout:6/716: rename d10/l7d to d10/d19/d3b/ddb/lec 0 2024-03-11T19:56:33.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/724: write d1/d21/d35/f5c [65511,130870] 0 2024-03-11T19:56:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/717: mkdir d10/d16/d24/d2e/ded 0 2024-03-11T19:56:33.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/665: unlink dd/de/d49/d46/f4f 0 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/839: sync 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:8/666: creat dd/de/d24/d28/fcd x:0 0 0 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/840: write d6/d9/f8c [559818,111092] 0 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/763: sync 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/764: fsync d4/d9/dae/f94 0 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:1/771: sync 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/841: truncate d6/f5f 4994983 0 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/750: sync 2024-03-11T19:56:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:8/667: link dd/de/d1b/cc0 dd/de/d1b/d20/d56/da4/dca/cce 0 2024-03-11T19:56:33.281 INFO:tasks.workunit.client.0.smithi107.stdout:1/772: mkdir d6/d7e/d81/d27/d2c/d32/dc1/df7 0 2024-03-11T19:56:33.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/773: chown d6/l23 15982 1 2024-03-11T19:56:33.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/774: creat d6/d7e/da5/dc7/ff8 x:0 0 0 2024-03-11T19:56:33.284 INFO:tasks.workunit.client.0.smithi107.stdout:7/842: link d6/d9/d57/fa1 d6/da/d4d/f108 0 2024-03-11T19:56:33.284 INFO:tasks.workunit.client.0.smithi107.stdout:2/725: dread d1/d1a/f20 [0,4194304] 0 2024-03-11T19:56:33.284 INFO:tasks.workunit.client.0.smithi107.stdout:2/726: fsync d1/d1a/d1e/d55/fee 0 2024-03-11T19:56:33.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/843: creat d6/da/d16/d91/f109 x:0 0 0 2024-03-11T19:56:33.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/727: mknod d1/d1a/d1e/d41/cf2 0 2024-03-11T19:56:33.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/728: fdatasync d1/d21/f28 0 2024-03-11T19:56:33.291 INFO:tasks.workunit.client.0.smithi107.stdout:8/668: link dd/f58 dd/fcf 0 2024-03-11T19:56:33.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/718: rename d10/d16/d1f/d51/db7 to d10/dee 0 2024-03-11T19:56:33.300 INFO:tasks.workunit.client.0.smithi107.stdout:1/775: rename d6/d7e/d81/d27/d2c/fb1 to d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/dd6/ff9 0 2024-03-11T19:56:33.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/776: rename d6/d7e/d81/d27/d2c/d32/dc1/d95 to d6/d7e/d81/d27/dfa 0 2024-03-11T19:56:33.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/777: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3/ffb x:0 0 0 2024-03-11T19:56:33.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/778: fdatasync d6/d7e/d81/d27/d2c/d72/f75 0 2024-03-11T19:56:33.307 INFO:tasks.workunit.client.0.smithi107.stdout:1/779: link d6/d7e/da5/dc7/feb d6/d7e/da5/dd9/ffc 0 2024-03-11T19:56:33.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/780: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/ffd x:0 0 0 2024-03-11T19:56:33.318 INFO:tasks.workunit.client.0.smithi107.stdout:1/781: rename d6/d19 to d6/d7e/d81/d27/d2c/d32/dc1/dc9/dfe 0 2024-03-11T19:56:33.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/903: read d1/d10/d13/d6c/d2d/d4d/f71 [107707,43937] 0 2024-03-11T19:56:33.328 INFO:tasks.workunit.client.0.smithi107.stdout:3/904: mkdir d1/d10/d13/d6c/d2d/d4d/d91/dcc/d11f 0 2024-03-11T19:56:33.328 INFO:tasks.workunit.client.0.smithi107.stdout:3/905: dread - d1/dbb/de3/dfd/fed zero size 2024-03-11T19:56:33.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/862: dwrite da/fe7 [0,4194304] 0 2024-03-11T19:56:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/863: chown da/d14/d43/d7f/fec 231921844 1 2024-03-11T19:56:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/864: chown da/d14/d18/f76 20635638 1 2024-03-11T19:56:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/865: chown da/d14/d18/d1e/d9b/fb9 0 1 2024-03-11T19:56:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/906: symlink d1/d10/d13/d6c/d2d/d4d/d91/df0/l120 0 2024-03-11T19:56:33.332 INFO:tasks.workunit.client.0.smithi107.stdout:3/907: unlink d1/d10/d13/d6c/laf 0 2024-03-11T19:56:33.332 INFO:tasks.workunit.client.0.smithi107.stdout:3/908: fdatasync d1/d10/d13/d6c/d2d/d4d/f53 0 2024-03-11T19:56:33.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/909: truncate d1/d10/d13/d4a/d4c/d49/f54 753490 0 2024-03-11T19:56:33.335 INFO:tasks.workunit.client.0.smithi107.stdout:1/782: write d6/d28/f80 [478559,40464] 0 2024-03-11T19:56:33.338 INFO:tasks.workunit.client.0.smithi107.stdout:1/783: truncate d6/f15 3432057 0 2024-03-11T19:56:33.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/729: dwrite d1/d1a/f65 [0,4194304] 0 2024-03-11T19:56:33.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/669: dread dd/d13/d18/f1a [4194304,4194304] 0 2024-03-11T19:56:33.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/670: dread - dd/d13/d32/f89 zero size 2024-03-11T19:56:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:8/671: unlink dd/de/d1b/d20/c70 0 2024-03-11T19:56:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/730: rename d1/d38/d75/d76/fc7 to d1/d1d/d64/d97/dd1/ff3 0 2024-03-11T19:56:33.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/731: dread - d1/d21/dbb/fd0 zero size 2024-03-11T19:56:33.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/732: fdatasync d1/d1a/fa3 0 2024-03-11T19:56:33.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/733: fsync d1/d1a/d34/d4f/db3/fb7 0 2024-03-11T19:56:33.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/672: mkdir dd/d13/d18/dd0 0 2024-03-11T19:56:33.371 INFO:tasks.workunit.client.0.smithi107.stdout:6/719: dread d10/d16/d24/d2e/d9d/f92 [0,4194304] 0 2024-03-11T19:56:33.371 INFO:tasks.workunit.client.0.smithi107.stdout:0/751: dwrite de/d11/d23/d32/d34/d39/d3a/d72/fb2 [0,4194304] 0 2024-03-11T19:56:33.372 INFO:tasks.workunit.client.0.smithi107.stdout:4/758: dwrite d4/d7/d1f/d2b/d93/dc3/dd0/fbf [0,4194304] 0 2024-03-11T19:56:33.372 INFO:tasks.workunit.client.0.smithi107.stdout:4/759: chown d4/d7/db/db8/fc8 23854341 1 2024-03-11T19:56:33.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/752: readlink de/d11/d23/d24/d4f/l6d 0 2024-03-11T19:56:33.375 INFO:tasks.workunit.client.0.smithi107.stdout:5/866: dwrite da/d14/d43/f9a [0,4194304] 0 2024-03-11T19:56:33.375 INFO:tasks.workunit.client.0.smithi107.stdout:5/867: read - da/d14/d38/f87 zero size 2024-03-11T19:56:33.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/753: mkdir de/d11/d23/d32/d34/de6 0 2024-03-11T19:56:33.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/754: chown de/d11/d23/d24/d83/dc4 15743648 1 2024-03-11T19:56:33.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/673: read dd/d13/f52 [646588,128749] 0 2024-03-11T19:56:33.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/674: creat dd/de/d1b/d2c/fd1 x:0 0 0 2024-03-11T19:56:33.412 INFO:tasks.workunit.client.0.smithi107.stdout:2/734: dwrite d1/d1a/d1e/d26/f62 [0,4194304] 0 2024-03-11T19:56:33.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/735: mkdir d1/d1a/d1e/d26/dd6/df4 0 2024-03-11T19:56:33.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/736: write d1/d21/d35/d67/f6e [89342,54062] 0 2024-03-11T19:56:33.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/737: chown d1/d1a/d34/d4f/c69 0 1 2024-03-11T19:56:33.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/738: chown d1/d1a/d1e/d26/f48 7712482 1 2024-03-11T19:56:33.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/720: dwrite d10/d16/fcd [0,4194304] 0 2024-03-11T19:56:33.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/721: write d10/d3d/fae [3351978,116942] 0 2024-03-11T19:56:33.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/722: getdents d10/d16/d2b/de2 0 2024-03-11T19:56:33.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/723: dread - d10/d19/d38/fb9 zero size 2024-03-11T19:56:33.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/755: rename de/d11/d23/d24/d4f/c64 to de/d11/d74/ce7 0 2024-03-11T19:56:33.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/756: chown de/d11/d23/d32/d34/d39/f53 10493 1 2024-03-11T19:56:33.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/757: unlink de/d11/d23/d32/f54 0 2024-03-11T19:56:33.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/758: rmdir de/d11/d23/d32/d34/d39/d3a/d72 39 2024-03-11T19:56:33.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/759: truncate de/f18 451374 0 2024-03-11T19:56:33.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/868: dwrite da/d14/d18/d44/dcd/fd5 [0,4194304] 0 2024-03-11T19:56:33.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/760: getdents de/d11/d74/dba 0 2024-03-11T19:56:33.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/761: chown de/d11/d23/d32/d34/d39/d55/l5c 3513482 1 2024-03-11T19:56:33.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/760: dwrite d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:56:33.435 INFO:tasks.workunit.client.0.smithi107.stdout:0/762: mkdir de/d11/de8 0 2024-03-11T19:56:33.437 INFO:tasks.workunit.client.0.smithi107.stdout:0/763: mknod de/d11/d23/d32/d34/d39/d55/d7c/ce9 0 2024-03-11T19:56:33.442 INFO:tasks.workunit.client.0.smithi107.stdout:0/764: mknod de/d11/d23/cea 0 2024-03-11T19:56:33.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/724: rename d10/d19/d38/d60 to d10/d16/d2b/d39/d6a/d6e/def 0 2024-03-11T19:56:33.453 INFO:tasks.workunit.client.0.smithi107.stdout:7/844: sync 2024-03-11T19:56:33.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/845: getdents d6/da/d16/d5c 0 2024-03-11T19:56:33.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/846: chown d6/da/d4d/f108 145005 1 2024-03-11T19:56:33.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/847: creat d6/da/d16/d5c/d7d/d60/d61/f10a x:0 0 0 2024-03-11T19:56:33.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/848: write d6/d9/d57/da2/fca [644529,62357] 0 2024-03-11T19:56:33.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/725: dread d10/d19/d3b/d70/dbb/fd6 [0,4194304] 0 2024-03-11T19:56:33.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/849: creat d6/da/d16/d5c/d7d/d1b/dfd/d9b/f10b x:0 0 0 2024-03-11T19:56:33.460 INFO:tasks.workunit.client.0.smithi107.stdout:6/726: mknod d10/d16/d1f/d51/d7a/cf0 0 2024-03-11T19:56:33.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/761: rename d4/d7/d1f/d2b/d24/d92 to d4/d7/d1f/d2b/d93/dc3/dd0/d79/df2 0 2024-03-11T19:56:33.470 INFO:tasks.workunit.client.0.smithi107.stdout:2/739: dwrite d1/d1a/d34/d4f/db3/fb7 [0,4194304] 0 2024-03-11T19:56:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:2/740: write d1/d1a/d8e/fec [856447,113448] 0 2024-03-11T19:56:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/762: creat d4/d7/d2f/d3b/ff3 x:0 0 0 2024-03-11T19:56:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/763: fsync d4/d7/d1f/d2b/d19/fa7 0 2024-03-11T19:56:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/764: write d4/d7/d1f/d2b/d19/fa2 [124677,64306] 0 2024-03-11T19:56:33.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/850: rename d6/l35 to d6/da/d16/l10c 0 2024-03-11T19:56:33.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/741: getdents d1/d21/d35 0 2024-03-11T19:56:33.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/851: write d6/da/d16/d5c/d7d/d60/fc3 [327295,61733] 0 2024-03-11T19:56:33.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/852: mkdir d6/d9/d57/d65/d10d 0 2024-03-11T19:56:33.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/853: chown d6/da/d4d/f71 2585 1 2024-03-11T19:56:33.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/854: getdents d6/da/d49/dee/dfc 0 2024-03-11T19:56:33.480 INFO:tasks.workunit.client.0.smithi107.stdout:7/855: rename d6/da/d16/d91/f109 to d6/da/d16/d5c/d7d/d1b/f10e 0 2024-03-11T19:56:33.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/856: unlink d6/da/d16/d5c/d7d/d1b/dfd/d9b/fe4 0 2024-03-11T19:56:33.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/857: getdents d6/d9/d57 0 2024-03-11T19:56:33.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/858: write d6/da/d16/d5c/d7d/d1b/f1c [517467,40559] 0 2024-03-11T19:56:33.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/859: mknod d6/da/d16/d5c/d7d/df0/c10f 0 2024-03-11T19:56:33.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/860: link d6/da/d16/d5c/d7d/d60/d61/lfb d6/da/d49/dee/d104/l110 0 2024-03-11T19:56:33.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/727: read d10/d16/d2b/d39/d6a/d6e/def/db8/fcc [1186547,68043] 0 2024-03-11T19:56:33.494 INFO:tasks.workunit.client.0.smithi107.stdout:6/728: unlink d10/d3d/d54/fa2 0 2024-03-11T19:56:33.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/861: dread d6/da/d16/d5c/d7d/d60/d61/d78/f8d [0,4194304] 0 2024-03-11T19:56:33.497 INFO:tasks.workunit.client.0.smithi107.stdout:7/862: mknod d6/da/d16/d5c/d7d/d60/d61/d9c/c111 0 2024-03-11T19:56:33.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/869: dwrite da/d14/d18/ff0 [0,4194304] 0 2024-03-11T19:56:33.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/729: write d10/d19/d3b/d70/fd8 [841920,96116] 0 2024-03-11T19:56:33.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/730: chown d10/d19/d38/fb9 371 1 2024-03-11T19:56:33.499 INFO:tasks.workunit.client.0.smithi107.stdout:7/863: creat d6/d9/f112 x:0 0 0 2024-03-11T19:56:33.500 INFO:tasks.workunit.client.0.smithi107.stdout:6/731: symlink d10/d16/d1f/d51/d76/d7f/lf1 0 2024-03-11T19:56:33.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/732: chown d10/d16/d2b/d39/d6a/d6e 53480 1 2024-03-11T19:56:33.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/733: rmdir d10/d16/d1f/d51/d7a/dba 39 2024-03-11T19:56:33.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/734: truncate d10/d3d/f98 564850 0 2024-03-11T19:56:33.512 INFO:tasks.workunit.client.0.smithi107.stdout:6/735: truncate d10/d16/fde 987660 0 2024-03-11T19:56:33.521 INFO:tasks.workunit.client.0.smithi107.stdout:0/765: dwrite de/d58/fa7 [0,4194304] 0 2024-03-11T19:56:33.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/766: symlink de/d11/d74/da2/dd3/dd4/dbf/dd1/leb 0 2024-03-11T19:56:33.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/767: truncate de/d11/d23/f71 3327023 0 2024-03-11T19:56:33.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/768: write de/d11/f4c [1393596,129689] 0 2024-03-11T19:56:33.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/864: dwrite d6/d9/d57/d65/f99 [0,4194304] 0 2024-03-11T19:56:33.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/865: fsync d6/da/d16/d5c/f70 0 2024-03-11T19:56:33.538 INFO:tasks.workunit.client.0.smithi107.stdout:2/742: dwrite d1/d1d/f3e [0,4194304] 0 2024-03-11T19:56:33.538 INFO:tasks.workunit.client.0.smithi107.stdout:6/736: dread d10/d16/d24/d2e/d9d/fa8 [0,4194304] 0 2024-03-11T19:56:33.540 INFO:tasks.workunit.client.0.smithi107.stdout:6/737: fsync d10/d16/d1f/f28 0 2024-03-11T19:56:33.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/738: write d10/d16/d24/d2e/f34 [1907244,40811] 0 2024-03-11T19:56:33.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/739: write d10/d16/d1f/d51/d76/fdf [5046737,17449] 0 2024-03-11T19:56:33.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/740: write d10/d16/d1f/d51/d76/f88 [1042831,41415] 0 2024-03-11T19:56:33.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/741: readlink d10/d16/l1d 0 2024-03-11T19:56:33.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/742: symlink d10/d19/d3b/d70/dbb/lf2 0 2024-03-11T19:56:33.581 INFO:tasks.workunit.client.0.smithi107.stdout:2/743: dwrite d1/d1a/d1e/d26/f62 [0,4194304] 0 2024-03-11T19:56:33.581 INFO:tasks.workunit.client.0.smithi107.stdout:2/744: truncate d1/f10 5056060 0 2024-03-11T19:56:33.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/745: creat d1/d1a/d1e/d26/dd6/ff5 x:0 0 0 2024-03-11T19:56:33.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/746: symlink d1/d1a/d34/lf6 0 2024-03-11T19:56:33.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/747: chown d1/d1a/d34/d4f/lb1 0 1 2024-03-11T19:56:33.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/748: chown d1/d1d/f5b 1019765 1 2024-03-11T19:56:33.585 INFO:tasks.workunit.client.0.smithi107.stdout:0/769: dwrite de/f13 [0,4194304] 0 2024-03-11T19:56:33.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/749: truncate d1/d1d/d64/d97/ff0 5161768 0 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:2/750: creat d1/d1d/d64/d97/ff7 x:0 0 0 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:2/751: write d1/d1a/d1e/d41/f51 [1908124,84895] 0 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:0/770: mkdir de/d11/d74/dec 0 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:3/910: sync 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:8/675: sync 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/784: sync 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:9/765: sync 2024-03-11T19:56:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/785: mkdir d6/d7e/d81/d27/dfa/dff 0 2024-03-11T19:56:33.594 INFO:tasks.workunit.client.0.smithi107.stdout:8/676: rename dd/d13/d18/c23 to dd/d13/d18/dd0/cd2 0 2024-03-11T19:56:33.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/786: readlink d6/d7e/d81/d27/d2c/d32/d58/l5a 0 2024-03-11T19:56:33.596 INFO:tasks.workunit.client.0.smithi107.stdout:8/677: link dd/d13/d32/f89 dd/d13/d32/fd3 0 2024-03-11T19:56:33.597 INFO:tasks.workunit.client.0.smithi107.stdout:8/678: creat dd/d13/d18/fd4 x:0 0 0 2024-03-11T19:56:33.597 INFO:tasks.workunit.client.0.smithi107.stdout:8/679: fsync dd/d13/d32/d51/fb3 0 2024-03-11T19:56:33.598 INFO:tasks.workunit.client.0.smithi107.stdout:1/787: mknod d6/d7e/d81/d27/d2c/d32/c100 0 2024-03-11T19:56:33.599 INFO:tasks.workunit.client.0.smithi107.stdout:1/788: mknod d6/d7e/c101 0 2024-03-11T19:56:33.599 INFO:tasks.workunit.client.0.smithi107.stdout:1/789: creat d6/d86/dc0/f102 x:0 0 0 2024-03-11T19:56:33.603 INFO:tasks.workunit.client.0.smithi107.stdout:9/766: unlink d4/d1d/d1f/d28/d2c/d96/lce 0 2024-03-11T19:56:33.604 INFO:tasks.workunit.client.0.smithi107.stdout:8/680: write dd/fb6 [1542849,115982] 0 2024-03-11T19:56:33.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/866: dwrite d6/d9/d57/d65/fc6 [0,4194304] 0 2024-03-11T19:56:33.606 INFO:tasks.workunit.client.0.smithi107.stdout:8/681: creat dd/de/d24/d28/d47/dbb/fd5 x:0 0 0 2024-03-11T19:56:33.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/682: creat dd/d13/d18/dd0/fd6 x:0 0 0 2024-03-11T19:56:33.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/683: mkdir dd/d13/dd7 0 2024-03-11T19:56:33.621 INFO:tasks.workunit.client.0.smithi107.stdout:3/911: dread d1/d10/d13/d4a/d4c/d49/fb5 [0,4194304] 0 2024-03-11T19:56:33.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/912: readlink d1/d10/d21/d43/l105 0 2024-03-11T19:56:33.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/867: dread d6/d9/d4e/fa4 [0,4194304] 0 2024-03-11T19:56:33.626 INFO:tasks.workunit.client.0.smithi107.stdout:7/868: rename d6/f17 to d6/d9/d57/d65/d10d/f113 0 2024-03-11T19:56:33.626 INFO:tasks.workunit.client.0.smithi107.stdout:7/869: write d6/da/d16/f3f [4470175,69926] 0 2024-03-11T19:56:33.626 INFO:tasks.workunit.client.0.smithi107.stdout:7/870: dread - d6/da/d16/d5c/d7d/d60/ff2 zero size 2024-03-11T19:56:33.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/684: creat dd/de/d1b/d20/d56/da4/dca/fd8 x:0 0 0 2024-03-11T19:56:33.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/743: dwrite d10/d19/d3b/f99 [0,4194304] 0 2024-03-11T19:56:33.646 INFO:tasks.workunit.client.0.smithi107.stdout:7/871: dread d6/da/d49/f4b [0,4194304] 0 2024-03-11T19:56:33.660 INFO:tasks.workunit.client.0.smithi107.stdout:1/790: dwrite d6/d7e/d81/f4e [0,4194304] 0 2024-03-11T19:56:33.664 INFO:tasks.workunit.client.0.smithi107.stdout:9/767: dwrite d4/d1d/d1f/d28/d2c/f34 [0,4194304] 0 2024-03-11T19:56:33.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/685: dread dd/de/d1b/f1d [4194304,4194304] 0 2024-03-11T19:56:33.668 INFO:tasks.workunit.client.0.smithi107.stdout:9/768: symlink d4/d9/d9f/l10e 0 2024-03-11T19:56:33.668 INFO:tasks.workunit.client.0.smithi107.stdout:8/686: mkdir dd/de/d49/d46/dd9 0 2024-03-11T19:56:33.671 INFO:tasks.workunit.client.0.smithi107.stdout:9/769: readlink d4/d1c/lec 0 2024-03-11T19:56:33.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/687: link c8 dd/d13/d18/d81/cda 0 2024-03-11T19:56:33.672 INFO:tasks.workunit.client.0.smithi107.stdout:2/752: dwrite d1/d1a/d1e/d26/f68 [0,4194304] 0 2024-03-11T19:56:33.672 INFO:tasks.workunit.client.0.smithi107.stdout:0/771: dread fd [0,4194304] 0 2024-03-11T19:56:33.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/772: readlink de/d11/d74/da2/dd3/dd4/l87 0 2024-03-11T19:56:33.680 INFO:tasks.workunit.client.0.smithi107.stdout:7/872: dread d6/f2f [4194304,4194304] 0 2024-03-11T19:56:33.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/873: mkdir d6/d114 0 2024-03-11T19:56:33.683 INFO:tasks.workunit.client.0.smithi107.stdout:1/791: rename d6/d7e/d81/d27/d2c/d32/d58/l5a to d6/d7e/da5/dc7/l103 0 2024-03-11T19:56:33.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/792: fdatasync d6/d7e/d81/d27/f7a 0 2024-03-11T19:56:33.688 INFO:tasks.workunit.client.0.smithi107.stdout:1/793: mkdir d6/d7e/d81/d104 0 2024-03-11T19:56:33.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/753: rename d1/d1a/d1e/d26/l52 to d1/d1a/d1e/d55/lf8 0 2024-03-11T19:56:33.691 INFO:tasks.workunit.client.0.smithi107.stdout:1/794: unlink d6/d7e/d81/d27/d2c/d32/dc1/f63 0 2024-03-11T19:56:33.691 INFO:tasks.workunit.client.0.smithi107.stdout:2/754: symlink d1/d1a/d1e/d26/dd6/lf9 0 2024-03-11T19:56:33.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/874: link d6/cd1 d6/d9/d57/d65/c115 0 2024-03-11T19:56:33.694 INFO:tasks.workunit.client.0.smithi107.stdout:2/755: mkdir d1/d1a/d1e/d26/dd6/dfa 0 2024-03-11T19:56:33.694 INFO:tasks.workunit.client.0.smithi107.stdout:2/756: creat d1/d1a/d1e/d41/ffb x:0 0 0 2024-03-11T19:56:33.694 INFO:tasks.workunit.client.0.smithi107.stdout:1/795: mknod d6/d86/c105 0 2024-03-11T19:56:33.695 INFO:tasks.workunit.client.0.smithi107.stdout:8/688: dread dd/de/d1b/d2c/f2d [0,4194304] 0 2024-03-11T19:56:33.695 INFO:tasks.workunit.client.0.smithi107.stdout:8/689: fdatasync dd/de/d1b/d20/f6a 0 2024-03-11T19:56:33.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/744: dwrite d10/d16/d24/f5b [0,4194304] 0 2024-03-11T19:56:33.697 INFO:tasks.workunit.client.0.smithi107.stdout:5/870: sync 2024-03-11T19:56:33.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/765: sync 2024-03-11T19:56:33.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/766: write d4/d7/d1f/d2b/f56 [1537989,58245] 0 2024-03-11T19:56:33.698 INFO:tasks.workunit.client.0.smithi107.stdout:9/770: dread d4/d1d/d5c/d64/f84 [0,4194304] 0 2024-03-11T19:56:33.701 INFO:tasks.workunit.client.0.smithi107.stdout:2/757: truncate d1/d1a/d1e/d26/f48 862157 0 2024-03-11T19:56:33.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/871: link da/d14/d96/dc1/c101 da/d14/d38/d59/c110 0 2024-03-11T19:56:33.705 INFO:tasks.workunit.client.0.smithi107.stdout:6/745: rmdir d10/d16/d1f/d51/d76/d7f/dbd 39 2024-03-11T19:56:33.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/771: getdents d4/d1d/d1f 0 2024-03-11T19:56:33.707 INFO:tasks.workunit.client.0.smithi107.stdout:5/872: getdents da 0 2024-03-11T19:56:33.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/875: write d6/da/d49/f4b [567813,62154] 0 2024-03-11T19:56:33.716 INFO:tasks.workunit.client.0.smithi107.stdout:7/876: truncate d6/d9/d57/da2/fcc 4457365 0 2024-03-11T19:56:33.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/796: getdents d6 0 2024-03-11T19:56:33.718 INFO:tasks.workunit.client.0.smithi107.stdout:0/773: dwrite de/d11/f4c [0,4194304] 0 2024-03-11T19:56:33.739 INFO:tasks.workunit.client.0.smithi107.stdout:6/746: dread d10/d16/d24/d2e/f86 [0,4194304] 0 2024-03-11T19:56:33.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/747: write d10/d19/d3b/d70/dbb/fd6 [204167,181] 0 2024-03-11T19:56:33.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/748: chown d10/d16/d1f/d51/d76 3 1 2024-03-11T19:56:33.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/749: chown d10/d16/d1f/f28 16115 1 2024-03-11T19:56:33.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/690: dwrite dd/d13/f34 [0,4194304] 0 2024-03-11T19:56:33.751 INFO:tasks.workunit.client.0.smithi107.stdout:6/750: creat d10/d19/d3b/ddb/ff3 x:0 0 0 2024-03-11T19:56:33.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/758: dread d1/d21/d35/fc2 [0,4194304] 0 2024-03-11T19:56:33.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/759: truncate d1/d21/ff1 630592 0 2024-03-11T19:56:33.755 INFO:tasks.workunit.client.0.smithi107.stdout:8/691: link dd/d13/d18/d81/f94 dd/d13/dd7/fdb 0 2024-03-11T19:56:33.755 INFO:tasks.workunit.client.0.smithi107.stdout:8/692: readlink dd/dac/lba 0 2024-03-11T19:56:33.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/693: dread - dd/d13/d18/f9d zero size 2024-03-11T19:56:33.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/760: getdents d1/d1a 0 2024-03-11T19:56:33.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/761: fdatasync d1/d38/d75/d76/f98 0 2024-03-11T19:56:33.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/762: readlink d1/d1a/d34/d4f/l93 0 2024-03-11T19:56:33.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/763: fsync d1/d38/d75/f89 0 2024-03-11T19:56:33.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/764: truncate d1/f2b 5159676 0 2024-03-11T19:56:33.784 INFO:tasks.workunit.client.0.smithi107.stdout:9/772: dwrite d4/d1d/d1f/d28/f42 [0,4194304] 0 2024-03-11T19:56:33.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/767: dwrite d4/d7/db/db8/fea [4194304,4194304] 0 2024-03-11T19:56:33.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/774: dwrite de/d11/d23/d24/f69 [0,4194304] 0 2024-03-11T19:56:33.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/775: rmdir de/d9e 39 2024-03-11T19:56:33.791 INFO:tasks.workunit.client.0.smithi107.stdout:0/776: creat de/d11/d74/da2/dd3/dd4/dbf/fed x:0 0 0 2024-03-11T19:56:33.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/777: unlink de/l3b 0 2024-03-11T19:56:33.795 INFO:tasks.workunit.client.0.smithi107.stdout:0/778: write de/d11/d23/d24/da4/dae/dcd/fd9 [126194,44835] 0 2024-03-11T19:56:33.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/913: sync 2024-03-11T19:56:33.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/768: write d4/d7/d1f/d2b/d24/d26/d29/d36/f78 [51701,26562] 0 2024-03-11T19:56:33.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/769: unlink d4/d7/c35 0 2024-03-11T19:56:33.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/914: rmdir d1/d10/d13/d6c/d2d/d4d/d91 39 2024-03-11T19:56:33.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/915: dread - d1/dbb/de3/dfd/fed zero size 2024-03-11T19:56:33.803 INFO:tasks.workunit.client.0.smithi107.stdout:7/877: dwrite d6/da/d16/d5c/f70 [0,4194304] 0 2024-03-11T19:56:33.804 INFO:tasks.workunit.client.0.smithi107.stdout:7/878: fsync d6/d9/f112 0 2024-03-11T19:56:33.805 INFO:tasks.workunit.client.0.smithi107.stdout:3/916: chown d1/d10/d26/da6 0 1 2024-03-11T19:56:33.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/770: creat d4/d7/d2f/d33/d7b/dca/ff4 x:0 0 0 2024-03-11T19:56:33.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/917: read d1/d10/d4e/fde [2320279,128799] 0 2024-03-11T19:56:33.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/771: creat d4/d7/d1f/d2b/d24/d26/ff5 x:0 0 0 2024-03-11T19:56:33.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/873: dwrite da/d14/d96/f88 [0,4194304] 0 2024-03-11T19:56:33.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/772: creat d4/d7/d2f/ff6 x:0 0 0 2024-03-11T19:56:33.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/773: write d4/d7/d1f/d2b/d93/fe0 [555433,103978] 0 2024-03-11T19:56:33.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/918: creat d1/f121 x:0 0 0 2024-03-11T19:56:33.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/773: write d4/d1d/d1f/d28/f42 [3283884,37541] 0 2024-03-11T19:56:33.812 INFO:tasks.workunit.client.0.smithi107.stdout:9/774: readlink d4/d9/dae/l72 0 2024-03-11T19:56:33.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/694: dwrite dd/d13/d18/f1a [0,4194304] 0 2024-03-11T19:56:33.813 INFO:tasks.workunit.client.0.smithi107.stdout:8/695: chown dd/de/d1b/d20/d5e/fa1 110460035 1 2024-03-11T19:56:33.813 INFO:tasks.workunit.client.0.smithi107.stdout:8/696: stat dd/de/d24 0 2024-03-11T19:56:33.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/874: write da/d14/d18/d1e/d9b/fa4 [2103589,96704] 0 2024-03-11T19:56:33.815 INFO:tasks.workunit.client.0.smithi107.stdout:3/919: mkdir d1/d10/d21/d43/d4f/d122 0 2024-03-11T19:56:33.816 INFO:tasks.workunit.client.0.smithi107.stdout:9/775: unlink d4/d9/c7c 0 2024-03-11T19:56:33.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/875: rmdir da/d14/d43/d47 39 2024-03-11T19:56:33.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/779: read de/d58/f8b [1750306,79795] 0 2024-03-11T19:56:33.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/876: fdatasync da/f86 0 2024-03-11T19:56:33.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/920: getdents d1/d10/d4e/d73/d8d 0 2024-03-11T19:56:33.821 INFO:tasks.workunit.client.0.smithi107.stdout:9/776: creat d4/d1d/d4b/f10f x:0 0 0 2024-03-11T19:56:33.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/921: mknod d1/d10/d4e/d73/c123 0 2024-03-11T19:56:33.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/922: write d1/d10/d13/d6c/d5d/d76/ff7 [871409,107759] 0 2024-03-11T19:56:33.835 INFO:tasks.workunit.client.0.smithi107.stdout:2/765: dwrite d1/d21/dbb/fd0 [0,4194304] 0 2024-03-11T19:56:33.836 INFO:tasks.workunit.client.0.smithi107.stdout:7/879: dread d6/da/d16/d5c/d7d/d60/fde [0,4194304] 0 2024-03-11T19:56:33.854 INFO:tasks.workunit.client.0.smithi107.stdout:2/766: mkdir d1/d21/d35/dfc 0 2024-03-11T19:56:33.855 INFO:tasks.workunit.client.0.smithi107.stdout:7/880: dread d6/da/d16/f2c [0,4194304] 0 2024-03-11T19:56:33.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/767: unlink d1/d1d/f24 0 2024-03-11T19:56:33.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/697: rename dd/de/d1b/d2c to dd/d13/d18/d81/ddc 0 2024-03-11T19:56:33.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/698: chown dd/de/d49 84502 1 2024-03-11T19:56:33.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/780: write de/d11/f4c [1006057,62548] 0 2024-03-11T19:56:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/881: rename d6/d9/d4e to d6/d21/d116 0 2024-03-11T19:56:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/882: getdents d6/d114 0 2024-03-11T19:56:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/883: chown d6/da/d16/l5b 78066861 1 2024-03-11T19:56:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/884: chown d6/d9/d57/fa1 1715 1 2024-03-11T19:56:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/885: readlink d6/da/d16/d5c/d7d/l5d 0 2024-03-11T19:56:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/777: write d4/d1d/d5c/d97/fe2 [785623,51150] 0 2024-03-11T19:56:33.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/797: sync 2024-03-11T19:56:33.877 INFO:tasks.workunit.client.0.smithi107.stdout:1/798: stat d6/d7e/d81/d27/d2c/d32/d58 0 2024-03-11T19:56:33.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/781: rename de/d58/f7f to de/d11/fee 0 2024-03-11T19:56:33.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/778: mknod d4/d1c/c110 0 2024-03-11T19:56:33.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/782: truncate de/d9e/dc8/fe0 269038 0 2024-03-11T19:56:33.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/783: fdatasync de/d11/d74/da2/dd3/dd4/dbf/fed 0 2024-03-11T19:56:33.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/923: dwrite d1/d45/fbf [0,4194304] 0 2024-03-11T19:56:33.880 INFO:tasks.workunit.client.0.smithi107.stdout:3/924: creat d1/d45/d85/f124 x:0 0 0 2024-03-11T19:56:33.880 INFO:tasks.workunit.client.0.smithi107.stdout:9/779: symlink d4/d1d/d5c/d97/l111 0 2024-03-11T19:56:33.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/784: mkdir de/d11/d23/d32/d34/d39/d55/d7c/def 0 2024-03-11T19:56:33.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/785: chown de/l1c 12894736 1 2024-03-11T19:56:33.903 INFO:tasks.workunit.client.0.smithi107.stdout:5/877: dwrite da/d31/d42/f13 [0,4194304] 0 2024-03-11T19:56:33.905 INFO:tasks.workunit.client.0.smithi107.stdout:5/878: creat da/d14/d38/d59/d5f/f111 x:0 0 0 2024-03-11T19:56:33.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/879: mkdir da/d14/d43/dab/d112 0 2024-03-11T19:56:33.913 INFO:tasks.workunit.client.0.smithi107.stdout:9/780: read d4/d9/dae/f77 [509969,75783] 0 2024-03-11T19:56:33.913 INFO:tasks.workunit.client.0.smithi107.stdout:5/880: symlink da/d14/d18/d1e/l113 0 2024-03-11T19:56:33.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/881: write da/d14/d96/dc1/fdc [2229318,45010] 0 2024-03-11T19:56:33.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/781: truncate d4/d1d/d1f/d28/d2c/ff9 395189 0 2024-03-11T19:56:33.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/782: stat d4/d1d/d1f/d28/l66 0 2024-03-11T19:56:33.916 INFO:tasks.workunit.client.0.smithi107.stdout:8/699: dwrite dd/d13/d18/f1a [0,4194304] 0 2024-03-11T19:56:33.917 INFO:tasks.workunit.client.0.smithi107.stdout:8/700: write dd/f2e [1408722,56551] 0 2024-03-11T19:56:33.917 INFO:tasks.workunit.client.0.smithi107.stdout:8/701: read - dd/d13/d18/dd0/fd6 zero size 2024-03-11T19:56:33.917 INFO:tasks.workunit.client.0.smithi107.stdout:9/783: chown d4/d1d/d5c/d97/cd2 454 1 2024-03-11T19:56:33.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/702: symlink dd/d16/ldd 0 2024-03-11T19:56:33.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/703: fdatasync dd/d13/d18/f5c 0 2024-03-11T19:56:33.919 INFO:tasks.workunit.client.0.smithi107.stdout:5/882: rmdir da/d14/d38/d59 39 2024-03-11T19:56:33.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/704: read dd/d13/fc9 [561889,101656] 0 2024-03-11T19:56:33.921 INFO:tasks.workunit.client.0.smithi107.stdout:9/784: mknod d4/c112 0 2024-03-11T19:56:33.921 INFO:tasks.workunit.client.0.smithi107.stdout:5/883: unlink da/d31/ff1 0 2024-03-11T19:56:33.926 INFO:tasks.workunit.client.0.smithi107.stdout:0/786: write de/d11/d23/d32/d34/d39/d3a/f5e [443214,20721] 0 2024-03-11T19:56:33.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/925: dwrite d1/d10/d4e/d73/d8d/fc1 [0,4194304] 0 2024-03-11T19:56:33.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/785: read d4/d9/dae/f94 [79760,129972] 0 2024-03-11T19:56:33.931 INFO:tasks.workunit.client.0.smithi107.stdout:8/705: rename dd/de/d1b/d4b/ca7 to dd/de/d24/d28/d47/cde 0 2024-03-11T19:56:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:1/799: dwrite d6/d28/f80 [0,4194304] 0 2024-03-11T19:56:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:1/800: chown d6/d7e/d81/l1f 68 1 2024-03-11T19:56:33.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/801: symlink d6/d7e/d81/d27/l106 0 2024-03-11T19:56:33.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/926: symlink d1/dbb/de3/l125 0 2024-03-11T19:56:33.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/927: chown d1/d10/d21/d43/d48 756 1 2024-03-11T19:56:33.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/928: symlink d1/d10/d21/d43/d48/l126 0 2024-03-11T19:56:33.938 INFO:tasks.workunit.client.0.smithi107.stdout:9/786: rename d4/d1d/d1f/d28/d2c/ff9 to d4/d1d/d1f/d28/d2c/f113 0 2024-03-11T19:56:33.939 INFO:tasks.workunit.client.0.smithi107.stdout:8/706: mknod dd/cdf 0 2024-03-11T19:56:33.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/787: rename d4/d1c/d47/d8a/c10c to d4/d9/d4d/c114 0 2024-03-11T19:56:33.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/788: fsync d4/d1c/d37/f8b 0 2024-03-11T19:56:33.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/789: dread d4/d1d/d1f/d28/d2c/f113 [0,4194304] 0 2024-03-11T19:56:33.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/790: symlink d4/d1d/d1f/d78/d105/l115 0 2024-03-11T19:56:33.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/707: rmdir dd/de/d1b/d20 39 2024-03-11T19:56:33.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/884: dread da/fe7 [0,4194304] 0 2024-03-11T19:56:33.956 INFO:tasks.workunit.client.0.smithi107.stdout:8/708: rename dd/d13/d18/fd4 to dd/d13/d32/fe0 0 2024-03-11T19:56:33.957 INFO:tasks.workunit.client.0.smithi107.stdout:5/885: creat da/f114 x:0 0 0 2024-03-11T19:56:33.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/709: getdents dd/d13/d18/d81/ddc 0 2024-03-11T19:56:33.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/710: fdatasync dd/f21 0 2024-03-11T19:56:33.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/711: read dd/d13/d18/f1a [5418866,34989] 0 2024-03-11T19:56:33.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/712: unlink dd/de/d1b/d20/d5e/fa1 0 2024-03-11T19:56:33.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/713: chown dd/de/d1b/d20/d5e/f65 730305099 1 2024-03-11T19:56:33.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/714: mkdir dd/de/d24/d28/de1 0 2024-03-11T19:56:33.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/715: truncate dd/d13/d18/f9d 102737 0 2024-03-11T19:56:33.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/716: chown dd/d13/fc9 92634712 1 2024-03-11T19:56:33.968 INFO:tasks.workunit.client.0.smithi107.stdout:1/802: dwrite d6/d7e/d81/d27/d2c/d32/dc1/f9b [0,4194304] 0 2024-03-11T19:56:33.969 INFO:tasks.workunit.client.0.smithi107.stdout:9/791: dread d4/d9/f86 [0,4194304] 0 2024-03-11T19:56:33.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/717: link dd/d13/d18/d81/ddc/l84 dd/d13/d18/d81/ddc/le2 0 2024-03-11T19:56:33.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/792: creat d4/d1d/d5c/d97/f116 x:0 0 0 2024-03-11T19:56:33.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/751: sync 2024-03-11T19:56:33.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/752: symlink d10/d16/d1f/d51/d9c/db6/lf4 0 2024-03-11T19:56:33.983 INFO:tasks.workunit.client.0.smithi107.stdout:8/718: truncate dd/de/d49/f73 1880329 0 2024-03-11T19:56:33.983 INFO:tasks.workunit.client.0.smithi107.stdout:8/719: dread - dd/d16/f8e zero size 2024-03-11T19:56:33.986 INFO:tasks.workunit.client.0.smithi107.stdout:8/720: mknod dd/de/d1b/d20/d56/da4/dca/ce3 0 2024-03-11T19:56:33.986 INFO:tasks.workunit.client.0.smithi107.stdout:8/721: chown dd/d13/d18/f9d 105913926 1 2024-03-11T19:56:33.986 INFO:tasks.workunit.client.0.smithi107.stdout:8/722: stat dd/de/d1b/d4b/f82 0 2024-03-11T19:56:33.986 INFO:tasks.workunit.client.0.smithi107.stdout:8/723: fdatasync dd/d13/d32/fbe 0 2024-03-11T19:56:33.989 INFO:tasks.workunit.client.0.smithi107.stdout:8/724: rename dd/de/d1b/d20/d56/da4/dca/fd8 to dd/d13/d18/d81/ddc/fe4 0 2024-03-11T19:56:33.991 INFO:tasks.workunit.client.0.smithi107.stdout:8/725: stat dd/d13/d18/d81/ddc/d90 0 2024-03-11T19:56:33.991 INFO:tasks.workunit.client.0.smithi107.stdout:8/726: readlink dd/d13/d32/d51/l76 0 2024-03-11T19:56:33.992 INFO:tasks.workunit.client.0.smithi107.stdout:3/929: dread d1/d10/d21/d43/d4f/f104 [0,4194304] 0 2024-03-11T19:56:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/787: dwrite de/d11/d23/d24/f60 [0,4194304] 0 2024-03-11T19:56:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/930: mknod d1/d10/d13/d4a/d4c/d49/dca/c127 0 2024-03-11T19:56:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/727: chown dd/d13/d18/d81/ddc/l30 0 1 2024-03-11T19:56:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/728: write dd/d13/d32/fbe [2420879,111676] 0 2024-03-11T19:56:33.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/793: dread d4/d1d/d1f/d28/d5d/df2/fda [0,4194304] 0 2024-03-11T19:56:33.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/729: getdents dd/d13 0 2024-03-11T19:56:34.003 INFO:tasks.workunit.client.0.smithi107.stdout:8/730: unlink dd/d13/d18/d81/ddc/d90/c91 0 2024-03-11T19:56:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:8/731: write dd/d13/d32/f89 [353653,3213] 0 2024-03-11T19:56:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:8/732: fsync dd/d16/f19 0 2024-03-11T19:56:34.014 INFO:tasks.workunit.client.0.smithi107.stdout:8/733: getdents dd/de 0 2024-03-11T19:56:34.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/734: link dd/d13/d18/d81/ddc/c5f dd/de/d24/d28/d47/dbb/ce5 0 2024-03-11T19:56:34.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/803: dwrite d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/dd6/ff9 [0,4194304] 0 2024-03-11T19:56:34.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/788: rename de/f13 to de/d11/d23/d32/d34/d39/d55/d7c/def/ff0 0 2024-03-11T19:56:34.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/789: chown de/d11/d23/d32/d34/d39/d55/d7c/ldf 434300 1 2024-03-11T19:56:34.019 INFO:tasks.workunit.client.0.smithi107.stdout:3/931: dread d1/d10/d4e/d73/d8d/f95 [0,4194304] 0 2024-03-11T19:56:34.022 INFO:tasks.workunit.client.0.smithi107.stdout:3/932: truncate d1/d10/f57 8100387 0 2024-03-11T19:56:34.022 INFO:tasks.workunit.client.0.smithi107.stdout:3/933: chown d1/d10/d13/d4a/d4c/d49/c118 4104 1 2024-03-11T19:56:34.022 INFO:tasks.workunit.client.0.smithi107.stdout:3/934: chown d1/d10/d21/fda 20 1 2024-03-11T19:56:34.023 INFO:tasks.workunit.client.0.smithi107.stdout:3/935: mkdir d1/d10/d2a/d7e/d8b/d128 0 2024-03-11T19:56:34.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/804: rename d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/da3 to d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107 0 2024-03-11T19:56:34.038 INFO:tasks.workunit.client.0.smithi107.stdout:1/805: creat d6/d5b/f108 x:0 0 0 2024-03-11T19:56:34.039 INFO:tasks.workunit.client.0.smithi107.stdout:1/806: rename d6/d7e/d81/d27/d2c/d32/dc1 to d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107/d109 22 2024-03-11T19:56:34.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/807: write d6/d7e/d81/d27/d2c/d32/d58/fcc [416478,120374] 0 2024-03-11T19:56:34.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/808: fdatasync d6/f10 0 2024-03-11T19:56:34.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/809: chown d6/d7e/d81/l20 8688251 1 2024-03-11T19:56:34.042 INFO:tasks.workunit.client.0.smithi107.stdout:6/753: dwrite d10/d19/f65 [0,4194304] 0 2024-03-11T19:56:34.043 INFO:tasks.workunit.client.0.smithi107.stdout:1/810: rename d6/d28/f7c to d6/d28/f10a 0 2024-03-11T19:56:34.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/811: write d6/fd [205650,36250] 0 2024-03-11T19:56:34.050 INFO:tasks.workunit.client.0.smithi107.stdout:3/936: dread d1/d10/d13/f61 [4194304,4194304] 0 2024-03-11T19:56:34.050 INFO:tasks.workunit.client.0.smithi107.stdout:3/937: truncate d1/d10/d21/d43/d48/ffc 18710 0 2024-03-11T19:56:34.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/794: dwrite d4/d9/dae/fe3 [0,4194304] 0 2024-03-11T19:56:34.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/938: link d1/d10/d26/lc2 d1/d10/d4e/d73/l129 0 2024-03-11T19:56:34.058 INFO:tasks.workunit.client.0.smithi107.stdout:3/939: rmdir d1/d10/d13/d6c/d2d/d4d/d91/da5 39 2024-03-11T19:56:34.058 INFO:tasks.workunit.client.0.smithi107.stdout:9/795: mknod d4/d1d/d1f/d28/d2c/d96/dd3/c117 0 2024-03-11T19:56:34.060 INFO:tasks.workunit.client.0.smithi107.stdout:9/796: mkdir d4/d1d/d1f/d78/d118 0 2024-03-11T19:56:34.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/797: truncate d4/d1d/d1f/d28/d2c/fa5 2116599 0 2024-03-11T19:56:34.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/798: fdatasync d4/d1d/d1f/d28/d2c/f113 0 2024-03-11T19:56:34.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/790: dwrite de/d11/d23/d32/d34/d39/f9f [0,4194304] 0 2024-03-11T19:56:34.066 INFO:tasks.workunit.client.0.smithi107.stdout:0/791: truncate de/d11/d74/da2/dd3/dd4/f48 19303 0 2024-03-11T19:56:34.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/792: mknod de/d11/d23/d32/d34/cf1 0 2024-03-11T19:56:34.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/793: getdents de/d11/d23/d24/da4 0 2024-03-11T19:56:34.072 INFO:tasks.workunit.client.0.smithi107.stdout:6/754: link d10/d16/d24/d2e/f36 d10/d3d/ff5 0 2024-03-11T19:56:34.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/794: symlink de/d11/d23/d24/d83/lf2 0 2024-03-11T19:56:34.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/795: chown de/d11/d23/d32/d34/d39/d55/l5c 10 1 2024-03-11T19:56:34.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/735: dwrite dd/de/d49/d46/fa5 [0,4194304] 0 2024-03-11T19:56:34.078 INFO:tasks.workunit.client.0.smithi107.stdout:0/796: mknod de/d11/d74/da2/dd3/cf3 0 2024-03-11T19:56:34.079 INFO:tasks.workunit.client.0.smithi107.stdout:0/797: chown de/d11/d74/da2/dd3/cf3 18736204 1 2024-03-11T19:56:34.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/755: truncate d10/d16/d1f/d51/d76/fdf 3691734 0 2024-03-11T19:56:34.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/736: creat dd/d13/d18/d81/ddc/fe6 x:0 0 0 2024-03-11T19:56:34.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/737: write dd/d13/d18/d81/f8d [1072823,89093] 0 2024-03-11T19:56:34.084 INFO:tasks.workunit.client.0.smithi107.stdout:3/940: write d1/d10/d13/d6c/d2d/d4d/d91/f106 [155272,5922] 0 2024-03-11T19:56:34.085 INFO:tasks.workunit.client.0.smithi107.stdout:8/738: mkdir dd/de/d49/d46/de7 0 2024-03-11T19:56:34.085 INFO:tasks.workunit.client.0.smithi107.stdout:6/756: mkdir d10/d16/df6 0 2024-03-11T19:56:34.086 INFO:tasks.workunit.client.0.smithi107.stdout:8/739: write dd/d13/d18/d81/ddc/fd1 [882052,11820] 0 2024-03-11T19:56:34.089 INFO:tasks.workunit.client.0.smithi107.stdout:3/941: rename d1/d10/d13/d6c/d5d/l77 to d1/d10/d21/d43/d48/d98/l12a 0 2024-03-11T19:56:34.089 INFO:tasks.workunit.client.0.smithi107.stdout:4/774: sync 2024-03-11T19:56:34.089 INFO:tasks.workunit.client.0.smithi107.stdout:7/886: sync 2024-03-11T19:56:34.089 INFO:tasks.workunit.client.0.smithi107.stdout:7/887: chown d6/d21/c75 823383 1 2024-03-11T19:56:34.089 INFO:tasks.workunit.client.0.smithi107.stdout:3/942: write d1/d10/d13/d6c/d5d/f94 [4564503,96387] 0 2024-03-11T19:56:34.090 INFO:tasks.workunit.client.0.smithi107.stdout:8/740: getdents dd/d13/d18/d81 0 2024-03-11T19:56:34.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/768: sync 2024-03-11T19:56:34.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/775: truncate d4/d7/db/d5c/fc4 2404904 0 2024-03-11T19:56:34.092 INFO:tasks.workunit.client.0.smithi107.stdout:8/741: mkdir dd/d13/d18/d81/ddc/de8 0 2024-03-11T19:56:34.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/888: rename d6/da/ff4 to d6/da/d16/d5c/d7d/d1b/dfd/d9b/f117 0 2024-03-11T19:56:34.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/799: dread d4/d9/d4d/fd6 [0,4194304] 0 2024-03-11T19:56:34.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/889: chown d6/da/d16/d5c/led 12 1 2024-03-11T19:56:34.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/800: fsync d4/d1c/d37/f8b 0 2024-03-11T19:56:34.096 INFO:tasks.workunit.client.0.smithi107.stdout:2/769: symlink d1/d38/lfd 0 2024-03-11T19:56:34.096 INFO:tasks.workunit.client.0.smithi107.stdout:3/943: mkdir d1/d10/d13/d6c/d2d/d4d/d91/da5/d12b 0 2024-03-11T19:56:34.097 INFO:tasks.workunit.client.0.smithi107.stdout:8/742: symlink dd/d13/d18/le9 0 2024-03-11T19:56:34.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/801: mknod d4/d1c/c119 0 2024-03-11T19:56:34.099 INFO:tasks.workunit.client.0.smithi107.stdout:3/944: creat d1/d10/d4e/f12c x:0 0 0 2024-03-11T19:56:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:0/798: write de/d11/d23/d24/f2d [2189764,28935] 0 2024-03-11T19:56:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:0/799: truncate de/d11/d23/d32/d34/d39/f6e 900631 0 2024-03-11T19:56:34.108 INFO:tasks.workunit.client.0.smithi107.stdout:3/945: dread d1/d10/d4e/d73/fc3 [0,4194304] 0 2024-03-11T19:56:34.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/743: dread dd/de/d1b/d20/d5e/f65 [0,4194304] 0 2024-03-11T19:56:34.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/946: truncate d1/d10/d26/fdb 64551 0 2024-03-11T19:56:34.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/947: chown d1/d10/d21/cfe 1 1 2024-03-11T19:56:34.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/744: rmdir dd/d16 39 2024-03-11T19:56:34.116 INFO:tasks.workunit.client.0.smithi107.stdout:3/948: mkdir d1/d45/d85/d12d 0 2024-03-11T19:56:34.119 INFO:tasks.workunit.client.0.smithi107.stdout:3/949: getdents d1/d10/d21/d43/d48 0 2024-03-11T19:56:34.120 INFO:tasks.workunit.client.0.smithi107.stdout:3/950: symlink d1/d10/d13/d6c/d2d/d4d/d91/df0/l12e 0 2024-03-11T19:56:34.122 INFO:tasks.workunit.client.0.smithi107.stdout:3/951: truncate d1/d10/d4e/d73/fc3 408203 0 2024-03-11T19:56:34.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/757: read d10/d16/d1f/d51/d76/f93 [2257335,1961] 0 2024-03-11T19:56:34.137 INFO:tasks.workunit.client.0.smithi107.stdout:3/952: creat d1/d10/d21/d43/d82/f12f x:0 0 0 2024-03-11T19:56:34.137 INFO:tasks.workunit.client.0.smithi107.stdout:3/953: write d1/d10/d13/d6c/d5d/f94 [3160302,73190] 0 2024-03-11T19:56:34.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/758: creat d10/d16/d1f/d51/d76/ff7 x:0 0 0 2024-03-11T19:56:34.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/954: creat d1/d10/d13/d6c/d2d/d4d/d91/f130 x:0 0 0 2024-03-11T19:56:34.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/955: rmdir d1/d10/d26/d17 39 2024-03-11T19:56:34.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/956: stat d1/d10/d13/d4a 0 2024-03-11T19:56:34.147 INFO:tasks.workunit.client.0.smithi107.stdout:3/957: creat d1/d10/d21/f131 x:0 0 0 2024-03-11T19:56:34.148 INFO:tasks.workunit.client.0.smithi107.stdout:3/958: truncate d1/d10/d13/d4a/d4c/d49/fb5 962132 0 2024-03-11T19:56:34.153 INFO:tasks.workunit.client.0.smithi107.stdout:3/959: mknod d1/d10/d26/da6/c132 0 2024-03-11T19:56:34.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/960: write d1/d10/d21/d43/d82/f12f [879945,984] 0 2024-03-11T19:56:34.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/961: truncate d1/d10/d13/d6c/d5d/fff 2089110 0 2024-03-11T19:56:34.158 INFO:tasks.workunit.client.0.smithi107.stdout:2/770: dwrite d1/d21/dbb/fd0 [0,4194304] 0 2024-03-11T19:56:34.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/771: symlink d1/d1a/d1e/d41/lfe 0 2024-03-11T19:56:34.185 INFO:tasks.workunit.client.0.smithi107.stdout:4/776: dwrite d4/d7/d1f/d2b/d24/d26/d29/f64 [0,4194304] 0 2024-03-11T19:56:34.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/745: dwrite dd/f4c [0,4194304] 0 2024-03-11T19:56:34.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/746: stat dd/d13/c6d 0 2024-03-11T19:56:34.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/747: readlink dd/de/d24/d28/d47/l67 0 2024-03-11T19:56:34.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/748: write dd/dac/fb9 [228657,2619] 0 2024-03-11T19:56:34.189 INFO:tasks.workunit.client.0.smithi107.stdout:0/800: dwrite de/d11/d23/d32/d34/d39/d3a/d72/f8a [0,4194304] 0 2024-03-11T19:56:34.189 INFO:tasks.workunit.client.0.smithi107.stdout:0/801: dread - de/d11/d23/d32/faf zero size 2024-03-11T19:56:34.191 INFO:tasks.workunit.client.0.smithi107.stdout:0/802: mkdir de/d9e/dc8/df4 0 2024-03-11T19:56:34.191 INFO:tasks.workunit.client.0.smithi107.stdout:0/803: creat de/d9e/dc8/ff5 x:0 0 0 2024-03-11T19:56:34.193 INFO:tasks.workunit.client.0.smithi107.stdout:0/804: creat de/d11/d23/d24/ff6 x:0 0 0 2024-03-11T19:56:34.208 INFO:tasks.workunit.client.0.smithi107.stdout:6/759: dwrite d10/f14 [0,4194304] 0 2024-03-11T19:56:34.208 INFO:tasks.workunit.client.0.smithi107.stdout:6/760: chown d10/d16/d2b/d8d/fdd 728051007 1 2024-03-11T19:56:34.208 INFO:tasks.workunit.client.0.smithi107.stdout:9/802: dwrite d4/d9/d9f/fbc [0,4194304] 0 2024-03-11T19:56:34.210 INFO:tasks.workunit.client.0.smithi107.stdout:9/803: fdatasync d4/d1c/d37/fbb 0 2024-03-11T19:56:34.216 INFO:tasks.workunit.client.0.smithi107.stdout:2/772: dread d1/d1a/d34/d6a/f82 [0,4194304] 0 2024-03-11T19:56:34.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/962: dwrite d1/d45/f86 [4194304,4194304] 0 2024-03-11T19:56:34.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/963: rename d1/d10/d4e/d73/d8d/d9c/f108 to d1/d10/d2a/d7e/d8b/f133 0 2024-03-11T19:56:34.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/964: mkdir d1/d10/d13/d4a/d4c/d49/dca/d134 0 2024-03-11T19:56:34.223 INFO:tasks.workunit.client.0.smithi107.stdout:5/886: sync 2024-03-11T19:56:34.225 INFO:tasks.workunit.client.0.smithi107.stdout:5/887: symlink da/d14/d18/d44/l115 0 2024-03-11T19:56:34.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/888: mknod da/d14/d38/d59/de5/c116 0 2024-03-11T19:56:34.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/889: mkdir da/d14/d18/d117 0 2024-03-11T19:56:34.232 INFO:tasks.workunit.client.0.smithi107.stdout:5/890: creat da/d14/d18/dff/d105/f118 x:0 0 0 2024-03-11T19:56:34.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/891: creat da/d14/d18/d44/da5/f119 x:0 0 0 2024-03-11T19:56:34.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/892: symlink da/d14/d38/d46/dd9/l11a 0 2024-03-11T19:56:34.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/893: symlink da/d14/d18/d44/l11b 0 2024-03-11T19:56:34.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/805: dwrite de/d7a/fb7 [0,4194304] 0 2024-03-11T19:56:34.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/806: stat f5 0 2024-03-11T19:56:34.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/894: mkdir da/d14/d18/d1e/d9b/d72/dc8/d11c 0 2024-03-11T19:56:34.242 INFO:tasks.workunit.client.0.smithi107.stdout:0/807: mknod de/d11/d74/da2/dd3/dd4/cf7 0 2024-03-11T19:56:34.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/804: write d4/d1c/f9e [1630286,109318] 0 2024-03-11T19:56:34.244 INFO:tasks.workunit.client.0.smithi107.stdout:8/749: dwrite dd/de/d1b/d4b/f82 [0,4194304] 0 2024-03-11T19:56:34.248 INFO:tasks.workunit.client.0.smithi107.stdout:9/805: mkdir d4/d11a 0 2024-03-11T19:56:34.248 INFO:tasks.workunit.client.0.smithi107.stdout:9/806: truncate d4/d1d/d1f/d28/d2c/d96/fe5 1309623 0 2024-03-11T19:56:34.248 INFO:tasks.workunit.client.0.smithi107.stdout:9/807: truncate d4/f54 5233842 0 2024-03-11T19:56:34.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/761: dwrite d10/d19/f31 [0,4194304] 0 2024-03-11T19:56:34.274 INFO:tasks.workunit.client.0.smithi107.stdout:6/762: readlink d10/d19/l35 0 2024-03-11T19:56:34.274 INFO:tasks.workunit.client.0.smithi107.stdout:6/763: fdatasync d10/d16/d24/f5b 0 2024-03-11T19:56:34.274 INFO:tasks.workunit.client.0.smithi107.stdout:6/764: truncate d10/d16/d2b/d39/dca/fea 877190 0 2024-03-11T19:56:34.275 INFO:tasks.workunit.client.0.smithi107.stdout:0/808: dwrite de/d9e/dc8/ff5 [0,4194304] 0 2024-03-11T19:56:34.277 INFO:tasks.workunit.client.0.smithi107.stdout:0/809: read de/f16 [91517,84984] 0 2024-03-11T19:56:34.278 INFO:tasks.workunit.client.0.smithi107.stdout:6/765: rmdir d10/d16/d2b/d39 39 2024-03-11T19:56:34.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/810: chown de/d11/d74/da2/dd3/dd4/dbf 316703821 1 2024-03-11T19:56:34.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/895: rename da/d14/d38/d59/de5 to da/d14/d18/d1e/d9b/d72/dc8/d11c/d11d 0 2024-03-11T19:56:34.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/766: write d10/d16/d1f/d51/d76/d7f/dbd/fda [859640,114329] 0 2024-03-11T19:56:34.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/808: dread d4/d9/dae/d6d/db0/fc2 [0,4194304] 0 2024-03-11T19:56:34.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/809: read d4/fe [4314093,54418] 0 2024-03-11T19:56:34.282 INFO:tasks.workunit.client.0.smithi107.stdout:0/811: rmdir de/d9e/dc8 39 2024-03-11T19:56:34.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/810: creat d4/d1d/d1f/f11b x:0 0 0 2024-03-11T19:56:34.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/811: chown d4/d1c/d47/d8a/caf 2660 1 2024-03-11T19:56:34.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/767: creat d10/dee/ff8 x:0 0 0 2024-03-11T19:56:34.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/750: dwrite dd/de/d1b/d20/d5e/fc7 [0,4194304] 0 2024-03-11T19:56:34.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/751: readlink dd/d13/d32/d51/l5d 0 2024-03-11T19:56:34.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/812: dread de/f6a [0,4194304] 0 2024-03-11T19:56:34.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/812: dread d4/d9/dae/fe3 [0,4194304] 0 2024-03-11T19:56:34.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/768: read d10/d16/d24/d2e/d9d/fa8 [2402742,54870] 0 2024-03-11T19:56:34.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/769: chown d10/d16/d24/d2e 263060 1 2024-03-11T19:56:34.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/770: mknod d10/d16/d2b/d39/dca/cf9 0 2024-03-11T19:56:34.312 INFO:tasks.workunit.client.0.smithi107.stdout:9/813: unlink d4/d1c/d37/dd9/de9/cea 0 2024-03-11T19:56:34.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/771: truncate d10/d16/d24/d2e/f2f 71674 0 2024-03-11T19:56:34.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/814: mknod d4/d1c/d37/dd9/de9/df8/c11c 0 2024-03-11T19:56:34.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/815: rmdir d4/d1d/d4b 39 2024-03-11T19:56:34.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/816: readlink d4/d1d/d1f/d28/d5d/lb4 0 2024-03-11T19:56:34.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/817: getdents d4/d1d/d4b/dfd 0 2024-03-11T19:56:34.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/818: write d4/d9/d4d/fd4 [244316,64701] 0 2024-03-11T19:56:34.317 INFO:tasks.workunit.client.0.smithi107.stdout:9/819: mknod d4/d11a/c11d 0 2024-03-11T19:56:34.317 INFO:tasks.workunit.client.0.smithi107.stdout:9/820: chown d4/d1d/d5c/d64/db3 13606 1 2024-03-11T19:56:34.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/752: dwrite dd/d13/f34 [0,4194304] 0 2024-03-11T19:56:34.318 INFO:tasks.workunit.client.0.smithi107.stdout:9/821: mknod d4/d1d/d5c/d64/c11e 0 2024-03-11T19:56:34.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/822: creat d4/d1c/d37/dd9/de9/f11f x:0 0 0 2024-03-11T19:56:34.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/823: readlink d4/d1d/d1f/l69 0 2024-03-11T19:56:34.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/824: creat d4/f120 x:0 0 0 2024-03-11T19:56:34.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/825: symlink d4/d1c/d37/dd9/de9/df8/l121 0 2024-03-11T19:56:34.348 INFO:tasks.workunit.client.0.smithi107.stdout:6/772: dwrite d10/d16/d1f/d51/d76/d7f/dbd/fda [0,4194304] 0 2024-03-11T19:56:34.373 INFO:tasks.workunit.client.0.smithi107.stdout:8/753: dwrite dd/d13/d32/d51/f85 [0,4194304] 0 2024-03-11T19:56:34.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/754: dwrite dd/d13/d18/d81/ddc/fd1 [0,4194304] 0 2024-03-11T19:56:34.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/755: creat dd/d13/dd7/fea x:0 0 0 2024-03-11T19:56:34.432 INFO:tasks.workunit.client.0.smithi107.stdout:8/756: read dd/d13/f34 [1856681,42708] 0 2024-03-11T19:56:34.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/757: unlink dd/de/d1b/d20/l9f 0 2024-03-11T19:56:34.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/758: write dd/d16/f33 [827508,113494] 0 2024-03-11T19:56:34.434 INFO:tasks.workunit.client.0.smithi107.stdout:8/759: creat dd/de/d49/d46/dd9/feb x:0 0 0 2024-03-11T19:56:34.444 INFO:tasks.workunit.client.0.smithi107.stdout:8/760: dread dd/d13/d18/f9d [0,4194304] 0 2024-03-11T19:56:34.444 INFO:tasks.workunit.client.0.smithi107.stdout:8/761: fsync dd/fb4 0 2024-03-11T19:56:34.473 INFO:tasks.workunit.client.0.smithi107.stdout:8/762: dread dd/d13/d32/d51/f61 [0,4194304] 0 2024-03-11T19:56:34.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/763: symlink dd/de/d1b/d20/d5e/dc8/lec 0 2024-03-11T19:56:34.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/764: read - dd/de/d1b/f9a zero size 2024-03-11T19:56:34.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/765: read dd/d13/d18/f75 [225336,12475] 0 2024-03-11T19:56:34.582 INFO:tasks.workunit.client.0.smithi107.stdout:5/896: rename da/d14/d18/d1e/d9b/d73/fae to da/d14/d18/dff/d105/d10f/f11e 0 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/965: sync 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/890: sync 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/812: sync 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:4/777: sync 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:4/778: chown d4/d7/d1f/d2b/d24 111 1 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/891: fsync d6/da/d16/f36 0 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:4/779: stat d4/d7/f22 0 2024-03-11T19:56:34.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/892: fdatasync d6/d21/d116/ff7 0 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/893: readlink d6/da/d16/d5c/d7d/dc5/lce 0 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/773: sync 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:5/897: creat da/d14/d18/d44/dcd/f11f x:0 0 0 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:5/898: fdatasync da/d14/d18/d6d/fa7 0 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/766: sync 2024-03-11T19:56:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/780: rmdir d4/d7/da0/dcb 39 2024-03-11T19:56:34.585 INFO:tasks.workunit.client.0.smithi107.stdout:9/826: sync 2024-03-11T19:56:34.585 INFO:tasks.workunit.client.0.smithi107.stdout:9/827: fdatasync d4/f54 0 2024-03-11T19:56:34.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/894: mkdir d6/d114/d118 0 2024-03-11T19:56:34.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/774: link d1/d1a/d1e/d55/f87 d1/d1a/d1e/d26/fff 0 2024-03-11T19:56:34.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/781: symlink d4/d7/d1f/d2b/d24/d26/d29/d36/lf7 0 2024-03-11T19:56:34.591 INFO:tasks.workunit.client.0.smithi107.stdout:0/813: rename lb to de/d11/d23/d24/d4f/d89/lf8 0 2024-03-11T19:56:34.591 INFO:tasks.workunit.client.0.smithi107.stdout:0/814: readlink de/d11/d23/d24/da4/dae/le2 0 2024-03-11T19:56:34.594 INFO:tasks.workunit.client.0.smithi107.stdout:8/767: creat dd/de/d49/fed x:0 0 0 2024-03-11T19:56:34.594 INFO:tasks.workunit.client.0.smithi107.stdout:8/768: stat dd/de/c11 0 2024-03-11T19:56:34.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/782: mknod d4/cf8 0 2024-03-11T19:56:34.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/783: fsync d4/d7/da0/fb3 0 2024-03-11T19:56:34.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/895: unlink d6/d9/f8 0 2024-03-11T19:56:34.602 INFO:tasks.workunit.client.0.smithi107.stdout:7/896: mkdir d6/da/d119 0 2024-03-11T19:56:34.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/897: unlink d6/da/d16/d5c/d7d/d60/f93 0 2024-03-11T19:56:34.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/898: fdatasync d6/da/d4d/fd5 0 2024-03-11T19:56:34.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/899: creat d6/da/d16/d5c/d7d/d1b/dfd/f11a x:0 0 0 2024-03-11T19:56:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/900: truncate d6/d9/f85 1704933 0 2024-03-11T19:56:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/901: write d6/da/d4d/fd5 [40370,53524] 0 2024-03-11T19:56:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/902: chown d6/d9/d57/c87 646738194 1 2024-03-11T19:56:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/903: read d6/d9/d57/d65/f90 [154817,37581] 0 2024-03-11T19:56:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/904: mkdir d6/da/d16/d5c/d7d/dc5/d11b 0 2024-03-11T19:56:34.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/775: dread d1/d1a/d1e/d26/f48 [0,4194304] 0 2024-03-11T19:56:34.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/899: dread da/d14/d43/d47/fa2 [0,4194304] 0 2024-03-11T19:56:34.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/769: dread dd/f69 [0,4194304] 0 2024-03-11T19:56:34.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/770: mkdir dd/de/d1b/d20/d56/da4/dee 0 2024-03-11T19:56:34.630 INFO:tasks.workunit.client.0.smithi107.stdout:5/900: rmdir da/d14/d18 39 2024-03-11T19:56:34.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/771: creat dd/d13/d18/d81/fef x:0 0 0 2024-03-11T19:56:34.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/772: mknod dd/d13/d32/cf0 0 2024-03-11T19:56:34.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/773: dread - dd/de/d49/fed zero size 2024-03-11T19:56:34.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/784: dread d4/d7/d1f/d2b/d24/d26/d29/d36/f78 [0,4194304] 0 2024-03-11T19:56:34.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/776: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:56:34.641 INFO:tasks.workunit.client.0.smithi107.stdout:4/785: readlink d4/d7/d1f/d2b/d93/dc3/dd0/l55 0 2024-03-11T19:56:34.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/813: dread d6/d7e/d81/d27/d2c/d72/f8a [0,4194304] 0 2024-03-11T19:56:34.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/774: creat dd/d13/ff1 x:0 0 0 2024-03-11T19:56:34.642 INFO:tasks.workunit.client.0.smithi107.stdout:2/777: truncate d1/d1a/d34/d4f/f5d 7401836 0 2024-03-11T19:56:34.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/773: rename d10/d16/d1f/d51/d76/d7f/lf1 to d10/d16/d2b/d39/d6a/lfa 0 2024-03-11T19:56:34.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/814: unlink d6/d7e/d81/l52 0 2024-03-11T19:56:34.644 INFO:tasks.workunit.client.0.smithi107.stdout:2/778: mkdir d1/d1a/d34/d4f/d63/d100 0 2024-03-11T19:56:34.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/815: getdents d6/d7e/da5/dc7 0 2024-03-11T19:56:34.646 INFO:tasks.workunit.client.0.smithi107.stdout:8/775: mkdir dd/de/d24/d28/df2 0 2024-03-11T19:56:34.646 INFO:tasks.workunit.client.0.smithi107.stdout:9/828: rename d4/d1d/d1f/f11b to d4/d106/f122 0 2024-03-11T19:56:34.647 INFO:tasks.workunit.client.0.smithi107.stdout:2/779: symlink d1/d1a/d1e/d55/l101 0 2024-03-11T19:56:34.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/780: symlink d1/d21/d35/dfc/l102 0 2024-03-11T19:56:34.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/781: fsync d1/d1d/d64/d97/dd1/ff3 0 2024-03-11T19:56:34.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/782: fsync d1/f15 0 2024-03-11T19:56:34.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/783: fdatasync d1/d1d/f77 0 2024-03-11T19:56:34.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/784: mkdir d1/d1a/d1e/d103 0 2024-03-11T19:56:34.650 INFO:tasks.workunit.client.0.smithi107.stdout:6/774: creat d10/d16/d2b/d39/d6a/ffb x:0 0 0 2024-03-11T19:56:34.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/785: mkdir d1/d1a/d34/d6a/d104 0 2024-03-11T19:56:34.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/786: chown d1/d1a/d1e/c6d 0 1 2024-03-11T19:56:34.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/905: rename d6/da/d16/c3d to d6/d9/d57/da2/c11c 0 2024-03-11T19:56:34.653 INFO:tasks.workunit.client.0.smithi107.stdout:6/775: symlink d10/lfc 0 2024-03-11T19:56:34.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/776: chown d10/d16/d1f/cb4 3527 1 2024-03-11T19:56:34.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/906: unlink d6/da/d49/ffa 0 2024-03-11T19:56:34.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/966: dread d1/d10/d13/d6c/d2d/d4d/d91/f106 [0,4194304] 0 2024-03-11T19:56:34.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/967: fdatasync d1/d10/d13/d6c/d5d/fff 0 2024-03-11T19:56:34.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/968: chown d1/d10/d26/da6/c132 1135 1 2024-03-11T19:56:34.673 INFO:tasks.workunit.client.0.smithi107.stdout:4/786: dread d4/d7/db/f49 [0,4194304] 0 2024-03-11T19:56:34.677 INFO:tasks.workunit.client.0.smithi107.stdout:8/776: write dd/de/f3b [1788077,81312] 0 2024-03-11T19:56:34.677 INFO:tasks.workunit.client.0.smithi107.stdout:5/901: dwrite da/d31/f4f [4194304,4194304] 0 2024-03-11T19:56:34.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/969: creat d1/d10/d13/da1/f135 x:0 0 0 2024-03-11T19:56:34.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/970: fsync d1/d10/d13/d6c/d2d/d4d/d91/da5/fd5 0 2024-03-11T19:56:34.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/971: write d1/d10/d21/f114 [372634,44412] 0 2024-03-11T19:56:34.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/777: symlink dd/de/d24/d28/lf3 0 2024-03-11T19:56:34.684 INFO:tasks.workunit.client.0.smithi107.stdout:8/778: mknod dd/de/d24/d28/cf4 0 2024-03-11T19:56:34.685 INFO:tasks.workunit.client.0.smithi107.stdout:8/779: chown dd/d13/d32/f89 3 1 2024-03-11T19:56:34.685 INFO:tasks.workunit.client.0.smithi107.stdout:0/815: dwrite de/d11/f91 [0,4194304] 0 2024-03-11T19:56:34.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/787: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:56:34.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/787: mknod d4/d7/d1f/d2b/d24/d26/d29/d36/cf9 0 2024-03-11T19:56:34.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/902: getdents da/d14/d43/d9e/dad 0 2024-03-11T19:56:34.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/788: write d4/d7/d1f/d2b/d19/d75/d8e/ff0 [650351,88542] 0 2024-03-11T19:56:34.694 INFO:tasks.workunit.client.0.smithi107.stdout:3/972: getdents d1/d10/d13/d4a 0 2024-03-11T19:56:34.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/789: creat d4/d7/d2f/d3c/ffa x:0 0 0 2024-03-11T19:56:34.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/788: symlink d1/d38/l105 0 2024-03-11T19:56:34.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/816: unlink de/d11/d23/d32/d34/cf1 0 2024-03-11T19:56:34.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/973: truncate d1/d10/d2a/d7e/d8b/f97 1887498 0 2024-03-11T19:56:34.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/974: write d1/d10/d21/fda [348678,122942] 0 2024-03-11T19:56:34.701 INFO:tasks.workunit.client.0.smithi107.stdout:6/777: dread d10/d16/d1f/d51/d76/d7f/dbd/fda [0,4194304] 0 2024-03-11T19:56:34.701 INFO:tasks.workunit.client.0.smithi107.stdout:0/817: mknod de/d11/d74/da3/cf9 0 2024-03-11T19:56:34.702 INFO:tasks.workunit.client.0.smithi107.stdout:4/790: mknod d4/d7/d2f/d33/d7b/dca/cfb 0 2024-03-11T19:56:34.702 INFO:tasks.workunit.client.0.smithi107.stdout:2/789: rename d1/d21/dda to d1/d1a/d1e/d103/d106 0 2024-03-11T19:56:34.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/818: dread - de/d58/f61 zero size 2024-03-11T19:56:34.705 INFO:tasks.workunit.client.0.smithi107.stdout:4/791: link d4/d7/d1f/d2b/d24/d26/d29/d36/f78 d4/d7/d1f/d2b/d19/d75/d8e/de9/ffc 0 2024-03-11T19:56:34.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/790: creat d1/d1a/d34/d4f/d63/d100/f107 x:0 0 0 2024-03-11T19:56:34.710 INFO:tasks.workunit.client.0.smithi107.stdout:7/907: write d6/da/d16/f6f [3358418,48676] 0 2024-03-11T19:56:34.710 INFO:tasks.workunit.client.0.smithi107.stdout:6/778: rmdir d10/d16/d2b/d39/d6a/d6e/def/db8 39 2024-03-11T19:56:34.710 INFO:tasks.workunit.client.0.smithi107.stdout:7/908: dread - d6/d21/fcb zero size 2024-03-11T19:56:34.711 INFO:tasks.workunit.client.0.smithi107.stdout:6/779: symlink d10/d16/d2b/lfd 0 2024-03-11T19:56:34.712 INFO:tasks.workunit.client.0.smithi107.stdout:6/780: readlink d10/d16/d2b/d39/l81 0 2024-03-11T19:56:34.715 INFO:tasks.workunit.client.0.smithi107.stdout:4/792: write d4/d7/d2f/d3c/fc6 [29997,92334] 0 2024-03-11T19:56:34.715 INFO:tasks.workunit.client.0.smithi107.stdout:6/781: stat d10/d16/d2b/d39/d6a/d6e/def 0 2024-03-11T19:56:34.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/791: dread d1/d1a/fa3 [0,4194304] 0 2024-03-11T19:56:34.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/792: dread - d1/fe0 zero size 2024-03-11T19:56:34.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/793: creat d1/d1a/d34/d4f/d63/db6/f108 x:0 0 0 2024-03-11T19:56:34.716 INFO:tasks.workunit.client.0.smithi107.stdout:4/793: creat d4/d7/d1f/d2b/d19/d75/d8e/de9/ffd x:0 0 0 2024-03-11T19:56:34.716 INFO:tasks.workunit.client.0.smithi107.stdout:4/794: chown d4/d7/d1f/d2b/d24/d26/d29/d36/cd4 10715 1 2024-03-11T19:56:34.717 INFO:tasks.workunit.client.0.smithi107.stdout:6/782: getdents d10/d16/d24/d2e 0 2024-03-11T19:56:34.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/795: write d4/d7/d1f/d2b/d93/dc3/dd0/d79/faa [884391,114769] 0 2024-03-11T19:56:34.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/794: mknod d1/d1a/d1e/d26/dd6/df4/c109 0 2024-03-11T19:56:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:6/783: symlink d10/d16/d2b/d39/d6a/d6e/def/db8/lfe 0 2024-03-11T19:56:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:6/784: fsync d10/d19/d3b/d70/dbb/fd6 0 2024-03-11T19:56:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:2/795: mknod d1/d21/d35/dfc/c10a 0 2024-03-11T19:56:34.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/796: symlink d1/d38/d75/dd7/l10b 0 2024-03-11T19:56:34.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/797: creat d1/d38/d75/d76/f10c x:0 0 0 2024-03-11T19:56:34.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/796: rmdir d4/d7/d1f/d2b/d24/d26 39 2024-03-11T19:56:34.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/816: dwrite d6/d28/dc4/f91 [0,4194304] 0 2024-03-11T19:56:34.731 INFO:tasks.workunit.client.0.smithi107.stdout:7/909: dread d6/da/d16/fa5 [0,4194304] 0 2024-03-11T19:56:34.735 INFO:tasks.workunit.client.0.smithi107.stdout:5/903: dread da/d14/d96/f88 [0,4194304] 0 2024-03-11T19:56:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/780: read dd/d13/f3d [1871506,20890] 0 2024-03-11T19:56:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/904: mkdir da/d14/d18/d44/dcd/d120 0 2024-03-11T19:56:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/905: chown da/d14/d38/d59/d5f/dc9 339 1 2024-03-11T19:56:34.741 INFO:tasks.workunit.client.0.smithi107.stdout:5/906: mkdir da/d14/d43/d9e/d121 0 2024-03-11T19:56:34.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/781: creat dd/d13/d18/d81/ddc/ff5 x:0 0 0 2024-03-11T19:56:34.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/975: write d1/d10/d13/f61 [25865,9826] 0 2024-03-11T19:56:34.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/976: fsync d1/d10/d13/d4a/d4c/ff3 0 2024-03-11T19:56:34.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/977: symlink d1/d10/d21/d43/d48/d98/l136 0 2024-03-11T19:56:34.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/978: symlink d1/d10/d4e/d73/d8d/d9c/l137 0 2024-03-11T19:56:34.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/979: creat d1/d10/d4e/d73/f138 x:0 0 0 2024-03-11T19:56:34.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/980: write d1/d10/d13/d4a/d4c/ff3 [2930082,57160] 0 2024-03-11T19:56:34.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/782: creat dd/d13/d18/d81/ddc/d90/d96/ff6 x:0 0 0 2024-03-11T19:56:34.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/783: creat dd/dac/ff7 x:0 0 0 2024-03-11T19:56:34.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/784: chown dd/d13/f52 76991 1 2024-03-11T19:56:34.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/785: creat dd/dac/ff8 x:0 0 0 2024-03-11T19:56:34.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/785: write d10/d16/d2b/d8d/fdd [1303806,31013] 0 2024-03-11T19:56:34.758 INFO:tasks.workunit.client.0.smithi107.stdout:6/786: getdents d10/d19 0 2024-03-11T19:56:34.762 INFO:tasks.workunit.client.0.smithi107.stdout:4/797: write d4/d7/d1f/d2b/d24/d26/d29/f30 [4222039,74385] 0 2024-03-11T19:56:34.778 INFO:tasks.workunit.client.0.smithi107.stdout:8/786: dread dd/d16/f33 [0,4194304] 0 2024-03-11T19:56:34.778 INFO:tasks.workunit.client.0.smithi107.stdout:8/787: chown dd/de/d49/d7e/fae 77779141 1 2024-03-11T19:56:34.779 INFO:tasks.workunit.client.0.smithi107.stdout:2/798: dwrite d1/d21/d35/fc2 [0,4194304] 0 2024-03-11T19:56:34.779 INFO:tasks.workunit.client.0.smithi107.stdout:0/819: dwrite de/d11/d23/d24/f60 [0,4194304] 0 2024-03-11T19:56:34.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/799: link d1/d1a/f7c d1/d1a/d34/f10d 0 2024-03-11T19:56:34.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/800: truncate d1/f7b 970965 0 2024-03-11T19:56:34.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/801: write d1/d1a/d34/d4f/d63/db6/f108 [181903,30429] 0 2024-03-11T19:56:34.782 INFO:tasks.workunit.client.0.smithi107.stdout:2/802: write d1/f7b [1394541,28107] 0 2024-03-11T19:56:34.784 INFO:tasks.workunit.client.0.smithi107.stdout:6/787: link d10/d16/d2b/d39/d6a/d6e/def/fa9 d10/d16/d2b/d39/d6a/fff 0 2024-03-11T19:56:34.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/788: rename dd/de/d49/d46/fa5 to dd/de/d1b/ff9 0 2024-03-11T19:56:34.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/789: write dd/de/d49/d46/dd9/feb [538740,102522] 0 2024-03-11T19:56:34.785 INFO:tasks.workunit.client.0.smithi107.stdout:0/820: creat de/d11/d74/da3/ffa x:0 0 0 2024-03-11T19:56:34.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/821: symlink de/d11/d23/d24/d4f/d89/lfb 0 2024-03-11T19:56:34.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/822: chown de/d11/d23/d24/d83 4444455 1 2024-03-11T19:56:34.788 INFO:tasks.workunit.client.0.smithi107.stdout:8/790: rename dd/d13/d18/d81/ddc/la2 to dd/d13/d18/d81/ddc/d90/d96/lfa 0 2024-03-11T19:56:34.791 INFO:tasks.workunit.client.0.smithi107.stdout:0/823: mknod de/d11/d74/da3/cfc 0 2024-03-11T19:56:34.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/788: rmdir d10/d16/d1f/d51/d76 39 2024-03-11T19:56:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/789: unlink d10/d16/d1f/d51/d76/d7f/f97 0 2024-03-11T19:56:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/798: dread d4/fbe [0,4194304] 0 2024-03-11T19:56:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/790: chown d10/d16/d2b/d39/d6a/d6e/la0 42 1 2024-03-11T19:56:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/799: fdatasync d4/d7/d1f/d2b/d19/d75/fe6 0 2024-03-11T19:56:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/907: read da/d14/d38/d59/d5f/f6a [2303550,104433] 0 2024-03-11T19:56:34.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/824: mknod de/d11/d74/da2/dd3/cfd 0 2024-03-11T19:56:34.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/791: mknod d10/d16/d1f/d51/d76/d7f/c100 0 2024-03-11T19:56:34.801 INFO:tasks.workunit.client.0.smithi107.stdout:5/908: link da/d14/c6b da/c122 0 2024-03-11T19:56:34.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/825: write de/d11/f51 [1403904,27651] 0 2024-03-11T19:56:34.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/826: readlink de/d11/d23/l46 0 2024-03-11T19:56:34.802 INFO:tasks.workunit.client.0.smithi107.stdout:0/827: chown de/d11/d23/d24 402299753 1 2024-03-11T19:56:34.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/909: mknod da/d14/d18/d1e/d9b/d73/c123 0 2024-03-11T19:56:34.805 INFO:tasks.workunit.client.0.smithi107.stdout:0/828: unlink de/d11/d23/f71 0 2024-03-11T19:56:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/803: dread d1/d1a/f7c [0,4194304] 0 2024-03-11T19:56:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/804: fsync d1/d1a/d1e/d26/f7a 0 2024-03-11T19:56:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/805: fsync d1/d1a/d34/d4f/db3/fb7 0 2024-03-11T19:56:34.808 INFO:tasks.workunit.client.0.smithi107.stdout:9/829: sync 2024-03-11T19:56:34.809 INFO:tasks.workunit.client.0.smithi107.stdout:6/792: dread d10/d16/fab [0,4194304] 0 2024-03-11T19:56:34.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/830: read - d4/d1d/d5c/d97/fe0 zero size 2024-03-11T19:56:34.811 INFO:tasks.workunit.client.0.smithi107.stdout:6/793: symlink d10/d19/l101 0 2024-03-11T19:56:34.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/910: dwrite d6/da/d16/d5c/d7d/d1b/dfd/d9b/f10b [0,4194304] 0 2024-03-11T19:56:34.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/911: write d6/d9/f112 [381902,15356] 0 2024-03-11T19:56:34.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/912: fdatasync d6/d21/d116/f59 0 2024-03-11T19:56:34.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/791: dread dd/d13/d32/d51/f85 [0,4194304] 0 2024-03-11T19:56:34.818 INFO:tasks.workunit.client.0.smithi107.stdout:6/794: mkdir d10/d16/d1f/d51/d9c/db6/d102 0 2024-03-11T19:56:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/800: dread f3 [0,4194304] 0 2024-03-11T19:56:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/801: stat d4/d7/d1f/l40 0 2024-03-11T19:56:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/802: chown d4/d7/d1f/d2b/d93/dc3/dd0/d79/faa 22 1 2024-03-11T19:56:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/803: chown d4/d7/db/d5e/f9f 11880173 1 2024-03-11T19:56:34.821 INFO:tasks.workunit.client.0.smithi107.stdout:6/795: symlink d10/d16/d24/d2e/l103 0 2024-03-11T19:56:34.821 INFO:tasks.workunit.client.0.smithi107.stdout:7/913: creat d6/da/d16/d5c/d7d/f11d x:0 0 0 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/910: dread da/d14/d18/d1e/d9b/d73/faa [0,4194304] 0 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/911: write da/d14/d38/d59/fc3 [632651,11244] 0 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/912: chown da/d14/d43/d7f/fec 50 1 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/914: write d6/da/d16/d5c/d7d/d60/fba [1161651,52427] 0 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:4/804: rename d4/d7/db/c45 to d4/d7/db/db8/cfe 0 2024-03-11T19:56:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/915: fdatasync d6/da/d16/fa5 0 2024-03-11T19:56:34.825 INFO:tasks.workunit.client.0.smithi107.stdout:5/913: getdents da/d14/d38/d59/dbd 0 2024-03-11T19:56:34.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/805: creat d4/d7/d1f/d2b/d24/fff x:0 0 0 2024-03-11T19:56:34.828 INFO:tasks.workunit.client.0.smithi107.stdout:5/914: truncate da/d14/d43/d7f/fec 818098 0 2024-03-11T19:56:34.828 INFO:tasks.workunit.client.0.smithi107.stdout:5/915: write da/d14/d18/d1e/d9b/fb9 [1529914,74328] 0 2024-03-11T19:56:34.837 INFO:tasks.workunit.client.0.smithi107.stdout:5/916: getdents da/d14/d38/d59 0 2024-03-11T19:56:34.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/796: dread d10/d16/fde [0,4194304] 0 2024-03-11T19:56:34.837 INFO:tasks.workunit.client.0.smithi107.stdout:7/916: write d6/d9/d57/d65/fef [123324,19419] 0 2024-03-11T19:56:34.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/981: dwrite d1/d10/d2a/d7e/d8b/fe1 [0,4194304] 0 2024-03-11T19:56:34.840 INFO:tasks.workunit.client.0.smithi107.stdout:3/982: mknod d1/d10/d13/d6c/d2d/d4d/c139 0 2024-03-11T19:56:34.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/983: creat d1/dbb/de3/dfd/f13a x:0 0 0 2024-03-11T19:56:34.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/984: stat d1/d10/d2a 0 2024-03-11T19:56:34.849 INFO:tasks.workunit.client.0.smithi107.stdout:8/792: read dd/de/d1b/d20/d5e/fa9 [3937665,65711] 0 2024-03-11T19:56:34.849 INFO:tasks.workunit.client.0.smithi107.stdout:8/793: dread - dd/d13/d18/d81/ddc/ff5 zero size 2024-03-11T19:56:34.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/806: dwrite d1/d1a/d34/fb5 [0,4194304] 0 2024-03-11T19:56:34.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/917: write da/d14/f68 [701526,412] 0 2024-03-11T19:56:34.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/918: read da/d14/d18/d44/f10c [664109,17781] 0 2024-03-11T19:56:34.875 INFO:tasks.workunit.client.0.smithi107.stdout:9/831: dread d4/d9/dae/fe3 [0,4194304] 0 2024-03-11T19:56:34.875 INFO:tasks.workunit.client.0.smithi107.stdout:9/832: chown d4/d1d/d1f/ff0 1142271 1 2024-03-11T19:56:34.875 INFO:tasks.workunit.client.0.smithi107.stdout:2/807: symlink d1/d21/d35/d67/l10e 0 2024-03-11T19:56:34.877 INFO:tasks.workunit.client.0.smithi107.stdout:5/919: truncate da/d14/d38/d59/dbd/fdd 3000590 0 2024-03-11T19:56:34.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/920: readlink da/d14/d18/d1e/l113 0 2024-03-11T19:56:34.897 INFO:tasks.workunit.client.0.smithi107.stdout:1/817: sync 2024-03-11T19:56:34.898 INFO:tasks.workunit.client.0.smithi107.stdout:7/917: dwrite d6/da/d16/d5c/d7d/d1b/dfd/d9b/f117 [0,4194304] 0 2024-03-11T19:56:34.899 INFO:tasks.workunit.client.0.smithi107.stdout:1/818: rmdir d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7 39 2024-03-11T19:56:34.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/918: write d6/d21/f3a [196941,66111] 0 2024-03-11T19:56:34.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/919: dread - d6/da/d16/d5c/d7d/f11d zero size 2024-03-11T19:56:34.900 INFO:tasks.workunit.client.0.smithi107.stdout:8/794: dwrite dd/d16/f19 [0,4194304] 0 2024-03-11T19:56:34.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/795: dread - dd/d13/d18/d81/ddc/ff5 zero size 2024-03-11T19:56:34.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/796: creat dd/d13/dd7/ffb x:0 0 0 2024-03-11T19:56:34.901 INFO:tasks.workunit.client.0.smithi107.stdout:1/819: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/f10b x:0 0 0 2024-03-11T19:56:34.901 INFO:tasks.workunit.client.0.smithi107.stdout:1/820: fdatasync d6/d7e/d81/d27/d2c/d72/f75 0 2024-03-11T19:56:34.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/821: mkdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c 0 2024-03-11T19:56:34.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/822: fsync d6/d7e/d81/d27/d2c/d32/d58/fc8 0 2024-03-11T19:56:34.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/823: write d6/d7e/d81/d27/d2c/f6a [490689,118049] 0 2024-03-11T19:56:34.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/824: mkdir d6/d7e/da5/d10d 0 2024-03-11T19:56:34.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/797: mkdir dd/de/dfc 0 2024-03-11T19:56:34.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/985: dwrite d1/d10/d21/d43/d48/fe9 [0,4194304] 0 2024-03-11T19:56:34.907 INFO:tasks.workunit.client.0.smithi107.stdout:8/798: mknod dd/de/d24/d28/d47/cfd 0 2024-03-11T19:56:34.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/986: rename d1/d10/d21/f114 to d1/d10/d13/d4a/d4c/d49/dca/d134/f13b 0 2024-03-11T19:56:34.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/987: fsync d1/d10/d26/d17/fe2 0 2024-03-11T19:56:34.908 INFO:tasks.workunit.client.0.smithi107.stdout:1/825: write d6/f15 [3892024,26220] 0 2024-03-11T19:56:34.910 INFO:tasks.workunit.client.0.smithi107.stdout:1/826: rename d6/d7e/d81/d27/c56 to d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/c10e 0 2024-03-11T19:56:34.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/827: symlink d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/l10f 0 2024-03-11T19:56:34.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/828: write d6/d7e/d81/d27/d2c/d72/fa6 [4968983,113096] 0 2024-03-11T19:56:34.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/920: dread d6/da/d16/d5c/d7d/d60/d61/fb3 [0,4194304] 0 2024-03-11T19:56:34.913 INFO:tasks.workunit.client.0.smithi107.stdout:1/829: unlink d6/d7e/d81/c76 0 2024-03-11T19:56:34.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/921: read da/d14/d38/f3f [3894664,85763] 0 2024-03-11T19:56:34.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/922: dread - da/d14/d18/f95 zero size 2024-03-11T19:56:34.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/923: stat da/d14/d18/fb1 0 2024-03-11T19:56:34.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/830: link d6/d7e/d81/d27/d2c/f30 d6/d7e/d81/d27/d2c/d32/dc1/df7/f110 0 2024-03-11T19:56:34.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/831: rename d6 to d6/d7e/d111 22 2024-03-11T19:56:34.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/832: mknod d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82/c112 0 2024-03-11T19:56:34.918 INFO:tasks.workunit.client.0.smithi107.stdout:6/797: dwrite d10/d19/f5d [0,4194304] 0 2024-03-11T19:56:34.918 INFO:tasks.workunit.client.0.smithi107.stdout:1/833: read d6/d28/f10a [519116,85790] 0 2024-03-11T19:56:34.921 INFO:tasks.workunit.client.0.smithi107.stdout:3/988: dread d1/d10/d13/d4a/d4c/d49/fb5 [0,4194304] 0 2024-03-11T19:56:34.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/798: truncate d10/d19/f31 7242228 0 2024-03-11T19:56:34.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/799: truncate d10/d16/d1f/d51/d76/f9f 251328 0 2024-03-11T19:56:34.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/808: read d1/d21/d35/f54 [4494423,121973] 0 2024-03-11T19:56:34.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/989: mknod d1/d10/d13/d6c/d2d/d4d/d91/dcc/d11f/c13c 0 2024-03-11T19:56:34.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/990: readlink d1/d10/d13/d6c/d2d/d4d/lcf 0 2024-03-11T19:56:34.927 INFO:tasks.workunit.client.0.smithi107.stdout:5/924: mkdir da/d14/d18/d1e/d124 0 2024-03-11T19:56:34.931 INFO:tasks.workunit.client.0.smithi107.stdout:2/809: rename d1/d38/d75/d76/c85 to d1/d1a/d1e/d103/d106/c10f 0 2024-03-11T19:56:34.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/991: creat d1/d10/d26/d17/f13d x:0 0 0 2024-03-11T19:56:34.952 INFO:tasks.workunit.client.0.smithi107.stdout:1/834: dread d6/d7e/d81/d27/f47 [4194304,4194304] 0 2024-03-11T19:56:34.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/833: dwrite d4/d1d/f3b [0,4194304] 0 2024-03-11T19:56:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:6/800: dread d10/d19/f77 [0,4194304] 0 2024-03-11T19:56:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/834: truncate d4/d1d/d5c/d64/f9d 1873716 0 2024-03-11T19:56:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/799: dwrite dd/fb4 [0,4194304] 0 2024-03-11T19:56:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/800: chown dd/de/d1b/d20/d56 3 1 2024-03-11T19:56:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/835: chown d4/d1d/d1f/d28/d5d/l6b 3 1 2024-03-11T19:56:34.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/801: write dd/de/d49/fed [664766,87632] 0 2024-03-11T19:56:34.966 INFO:tasks.workunit.client.0.smithi107.stdout:8/802: read - dd/de/d24/d28/d47/dbb/fd5 zero size 2024-03-11T19:56:34.966 INFO:tasks.workunit.client.0.smithi107.stdout:5/925: write da/d14/d18/d1e/d9b/d73/fd6 [2437884,117182] 0 2024-03-11T19:56:34.968 INFO:tasks.workunit.client.0.smithi107.stdout:1/835: dread d6/d7e/d81/d27/d2c/d7d/dd8/ff1 [0,4194304] 0 2024-03-11T19:56:34.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/926: mknod da/d14/d38/d59/d5f/dc9/c125 0 2024-03-11T19:56:34.973 INFO:tasks.workunit.client.0.smithi107.stdout:1/836: creat d6/f113 x:0 0 0 2024-03-11T19:56:34.973 INFO:tasks.workunit.client.0.smithi107.stdout:1/837: chown d6/f68 750854 1 2024-03-11T19:56:34.975 INFO:tasks.workunit.client.0.smithi107.stdout:1/838: link d6/d7e/d81/d27/d2c/c92 d6/d86/c114 0 2024-03-11T19:56:34.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/839: truncate d6/d28/f4b 1863531 0 2024-03-11T19:56:34.980 INFO:tasks.workunit.client.0.smithi107.stdout:2/810: dread d1/d21/f96 [0,4194304] 0 2024-03-11T19:56:34.980 INFO:tasks.workunit.client.0.smithi107.stdout:2/811: write d1/d1d/d64/fe3 [4480698,123549] 0 2024-03-11T19:56:34.983 INFO:tasks.workunit.client.0.smithi107.stdout:2/812: rename d1/d1a/d1e/d26/c56 to d1/d38/d75/c110 0 2024-03-11T19:56:34.986 INFO:tasks.workunit.client.0.smithi107.stdout:7/921: dwrite d6/d9/f6d [0,4194304] 0 2024-03-11T19:56:34.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/922: mknod d6/da/d16/d5c/d7d/dc5/c11e 0 2024-03-11T19:56:34.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/923: mknod d6/da/d16/d91/dc8/d80/ddd/c11f 0 2024-03-11T19:56:34.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/927: dread da/d14/d18/d44/f10c [0,4194304] 0 2024-03-11T19:56:34.996 INFO:tasks.workunit.client.0.smithi107.stdout:5/928: truncate da/d14/d43/d9e/dad/fb0 582585 0 2024-03-11T19:56:34.996 INFO:tasks.workunit.client.0.smithi107.stdout:5/929: chown da/d14/d38/d59/dbd/fdd 178381 1 2024-03-11T19:56:34.997 INFO:tasks.workunit.client.0.smithi107.stdout:7/924: unlink d6/ldc 0 2024-03-11T19:56:34.998 INFO:tasks.workunit.client.0.smithi107.stdout:7/925: creat d6/d114/d118/f120 x:0 0 0 2024-03-11T19:56:35.002 INFO:tasks.workunit.client.0.smithi107.stdout:7/926: symlink d6/da/d16/d5c/d7d/d60/d61/d78/l121 0 2024-03-11T19:56:35.002 INFO:tasks.workunit.client.0.smithi107.stdout:7/927: fdatasync d6/d9/f85 0 2024-03-11T19:56:35.002 INFO:tasks.workunit.client.0.smithi107.stdout:7/928: read d6/d21/d116/fa4 [381634,55474] 0 2024-03-11T19:56:35.002 INFO:tasks.workunit.client.0.smithi107.stdout:7/929: truncate d6/da/d16/d5c/f4a 2081651 0 2024-03-11T19:56:35.003 INFO:tasks.workunit.client.0.smithi107.stdout:7/930: symlink d6/d9/l122 0 2024-03-11T19:56:35.004 INFO:tasks.workunit.client.0.smithi107.stdout:3/992: dwrite d1/f121 [0,4194304] 0 2024-03-11T19:56:35.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/836: dread d4/d1d/d1f/d28/d2c/fa3 [0,4194304] 0 2024-03-11T19:56:35.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/837: dread - d4/f120 zero size 2024-03-11T19:56:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/993: symlink d1/d10/d2a/d7e/d8b/d128/l13e 0 2024-03-11T19:56:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/994: dread - d1/d10/d13/d6c/d2d/d4d/d91/dcd/f116 zero size 2024-03-11T19:56:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/995: dread - d1/d10/d13/d6c/d2d/d4d/d91/f11b zero size 2024-03-11T19:56:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/801: dread d10/d16/d1f/d51/d9c/fb0 [0,4194304] 0 2024-03-11T19:56:35.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:34 smithi122 ceph-mon[36397]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 71 op/s 2024-03-11T19:56:35.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/802: creat d10/d16/d1f/d51/d9c/f104 x:0 0 0 2024-03-11T19:56:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/838: mknod d4/d1d/d1f/d28/d2c/c123 0 2024-03-11T19:56:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/839: write d4/d1d/d5c/d64/db3/fc8 [844287,79594] 0 2024-03-11T19:56:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/840: dread - d4/d9/d9f/dd8/fe4 zero size 2024-03-11T19:56:35.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/841: rename d4/d1d/d1f/d78/f1a to d4/d9/f124 0 2024-03-11T19:56:35.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/842: mkdir d4/d1d/d5c/d97/d125 0 2024-03-11T19:56:35.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/843: truncate d4/d1d/d5c/d64/fa8 2021140 0 2024-03-11T19:56:35.029 INFO:tasks.workunit.client.0.smithi107.stdout:5/930: link da/d14/d18/d1e/c34 da/d14/d18/d1e/d9b/d72/dc8/c126 0 2024-03-11T19:56:35.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/931: rename da/d31/ce0 to da/d14/d43/c127 0 2024-03-11T19:56:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:5/932: mknod da/d14/d18/dff/c128 0 2024-03-11T19:56:35.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/996: dread d1/d10/d2a/d7e/d8b/fe1 [0,4194304] 0 2024-03-11T19:56:35.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/997: readlink d1/d10/d4e/d10b/l10c 0 2024-03-11T19:56:35.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/998: fdatasync d1/d10/d13/d6c/d5d/d76/ff7 0 2024-03-11T19:56:35.033 INFO:tasks.workunit.client.0.smithi107.stdout:3/999: fsync d1/d10/d21/d43/d48/fe9 0 2024-03-11T19:56:35.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:34 smithi145 ceph-mon[36730]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 71 op/s 2024-03-11T19:56:35.034 INFO:tasks.workunit.client.0.smithi107.stdout:5/933: creat da/d14/d38/d59/d5f/f129 x:0 0 0 2024-03-11T19:56:35.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/934: getdents da 0 2024-03-11T19:56:35.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/935: unlink da/d14/d18/d1e/d9b/db3/ffc 0 2024-03-11T19:56:35.040 INFO:tasks.workunit.client.0.smithi107.stdout:9/844: dread d4/d1d/d1f/d28/d2c/f82 [0,4194304] 0 2024-03-11T19:56:35.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:34 smithi107 ceph-mon[31712]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 71 op/s 2024-03-11T19:56:35.048 INFO:tasks.workunit.client.0.smithi107.stdout:6/803: dwrite d10/d16/d2b/d39/d6a/fd1 [0,4194304] 0 2024-03-11T19:56:35.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/936: rename da/d14/d18/d1e/d9b/d73/fa3 to da/d14/d18/d1e/d124/f12a 0 2024-03-11T19:56:35.058 INFO:tasks.workunit.client.0.smithi107.stdout:9/845: write d4/d9/d4d/f8e [1712109,126888] 0 2024-03-11T19:56:35.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/846: dread d4/d9/d4d/fd4 [0,4194304] 0 2024-03-11T19:56:35.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/847: truncate d4/d9/dae/d6d/f68 1033584 0 2024-03-11T19:56:35.068 INFO:tasks.workunit.client.0.smithi107.stdout:9/848: rename d4/d9/dae/d6d/db0/fc2 to d4/d1d/d1f/d78/d118/f126 0 2024-03-11T19:56:35.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/804: read d10/d19/f31 [4005099,23745] 0 2024-03-11T19:56:35.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/805: chown d10/d19/d3b/d70/dbb 10064660 1 2024-03-11T19:56:35.096 INFO:tasks.workunit.client.0.smithi107.stdout:6/806: write d10/d16/f84 [277630,86355] 0 2024-03-11T19:56:35.096 INFO:tasks.workunit.client.0.smithi107.stdout:6/807: stat d10/d3d/d54 0 2024-03-11T19:56:35.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/808: symlink d10/d3d/d54/l105 0 2024-03-11T19:56:35.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/809: write d10/d19/d38/fce [790276,67688] 0 2024-03-11T19:56:35.101 INFO:tasks.workunit.client.0.smithi107.stdout:6/810: mknod d10/d16/d2b/d8d/c106 0 2024-03-11T19:56:35.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/811: truncate d10/d3d/ff5 2590638 0 2024-03-11T19:56:35.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/812: fdatasync d10/d16/d2b/d39/f66 0 2024-03-11T19:56:35.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/937: dwrite f4 [0,4194304] 0 2024-03-11T19:56:35.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/938: chown da/d14/d43/d9e/dad/fb0 0 1 2024-03-11T19:56:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/849: dwrite d4/d1d/d1f/d28/d2c/d96/dd3/fdf [0,4194304] 0 2024-03-11T19:56:35.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/813: dread d10/d16/d1f/f25 [0,4194304] 0 2024-03-11T19:56:35.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/814: rmdir d10/d16/d2b/d39/d6a/d6e/def 39 2024-03-11T19:56:35.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/815: chown d10/d16/d2b/d39/dca 73966 1 2024-03-11T19:56:35.146 INFO:tasks.workunit.client.0.smithi107.stdout:6/816: dread d10/d16/d24/f68 [0,4194304] 0 2024-03-11T19:56:35.156 INFO:tasks.workunit.client.0.smithi107.stdout:5/939: dwrite da/d14/d18/d1e/d9b/d73/ff2 [0,4194304] 0 2024-03-11T19:56:35.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/940: mknod da/d14/d38/d59/d5f/dc9/c12b 0 2024-03-11T19:56:35.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/941: chown da/d14/d43/f9a 21014686 1 2024-03-11T19:56:35.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/850: dwrite d4/d1d/d1f/d28/d2c/fa3 [0,4194304] 0 2024-03-11T19:56:35.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/851: creat d4/d1d/d1f/d28/d2c/d96/f127 x:0 0 0 2024-03-11T19:56:35.174 INFO:tasks.workunit.client.0.smithi107.stdout:5/942: dread da/d14/d43/dab/fee [0,4194304] 0 2024-03-11T19:56:35.175 INFO:tasks.workunit.client.0.smithi107.stdout:5/943: creat da/d14/d43/d47/f12c x:0 0 0 2024-03-11T19:56:35.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/944: getdents da/d14/d43/d9e 0 2024-03-11T19:56:35.192 INFO:tasks.workunit.client.0.smithi107.stdout:6/817: dwrite d10/f13 [0,4194304] 0 2024-03-11T19:56:35.193 INFO:tasks.workunit.client.0.smithi107.stdout:5/945: dread da/d14/d96/dc1/fdc [0,4194304] 0 2024-03-11T19:56:35.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/946: write da/d14/d38/d59/dbd/fde [1792808,82976] 0 2024-03-11T19:56:35.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/947: write da/f85 [552655,70837] 0 2024-03-11T19:56:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/948: dread da/d31/d42/f23 [0,4194304] 0 2024-03-11T19:56:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:6/818: dwrite d10/d19/f31 [4194304,4194304] 0 2024-03-11T19:56:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/819: creat d10/d16/df6/f107 x:0 0 0 2024-03-11T19:56:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:6/820: creat d10/d16/d2b/d39/f108 x:0 0 0 2024-03-11T19:56:35.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/806: sync 2024-03-11T19:56:35.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/821: read d10/d3d/ff5 [489552,81958] 0 2024-03-11T19:56:35.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/949: write da/d14/d43/d7f/fc2 [2140977,48784] 0 2024-03-11T19:56:35.257 INFO:tasks.workunit.client.0.smithi107.stdout:5/950: mknod da/d14/d43/dab/c12d 0 2024-03-11T19:56:35.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/822: mkdir d10/d19/d38/d109 0 2024-03-11T19:56:35.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/823: chown d10/d16/d1f/d51/f87 29909 1 2024-03-11T19:56:35.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/951: mkdir da/d31/d12e 0 2024-03-11T19:56:35.259 INFO:tasks.workunit.client.0.smithi107.stdout:6/824: symlink d10/d16/d1f/d51/d9c/db6/l10a 0 2024-03-11T19:56:35.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/952: link da/d14/d18/d1e/d9b/d73/cbf da/d14/d43/dab/d112/c12f 0 2024-03-11T19:56:35.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/953: dread - da/d14/d18/d1e/d9b/d72/dc8/fe2 zero size 2024-03-11T19:56:35.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/954: symlink da/d14/d96/dc1/l130 0 2024-03-11T19:56:35.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/955: mknod da/d14/d38/d46/dd9/c131 0 2024-03-11T19:56:35.263 INFO:tasks.workunit.client.0.smithi107.stdout:5/956: creat da/d14/d96/dc1/f132 x:0 0 0 2024-03-11T19:56:35.265 INFO:tasks.workunit.client.0.smithi107.stdout:5/957: creat da/d14/d43/d9e/dad/f133 x:0 0 0 2024-03-11T19:56:35.265 INFO:tasks.workunit.client.0.smithi107.stdout:5/958: readlink da/d14/d38/d59/l5b 0 2024-03-11T19:56:35.265 INFO:tasks.workunit.client.0.smithi107.stdout:5/959: fdatasync da/d14/d38/fef 0 2024-03-11T19:56:35.266 INFO:tasks.workunit.client.0.smithi107.stdout:5/960: creat da/d14/d38/d46/f134 x:0 0 0 2024-03-11T19:56:35.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/961: unlink da/d14/d38/d59/d5f/c70 0 2024-03-11T19:56:35.269 INFO:tasks.workunit.client.0.smithi107.stdout:5/962: link da/d31/c66 da/d14/c135 0 2024-03-11T19:56:35.276 INFO:tasks.workunit.client.0.smithi107.stdout:4/807: dwrite d4/d7/da0/fb9 [0,4194304] 0 2024-03-11T19:56:35.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/808: read d4/d7/d2f/d3c/f73 [19811,17849] 0 2024-03-11T19:56:35.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/825: link d10/d19/d3b/d70/le3 d10/d16/d1f/d51/d76/l10b 0 2024-03-11T19:56:35.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/826: symlink d10/d16/d1f/d51/d76/l10c 0 2024-03-11T19:56:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:5/963: dwrite da/d31/f4f [4194304,4194304] 0 2024-03-11T19:56:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:5/964: chown da/d14/d38/d59/d5f/f80 189 1 2024-03-11T19:56:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:5/965: chown da/d14/d18/d6d/lb5 22993912 1 2024-03-11T19:56:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/827: link d10/d16/f2d d10/d16/d2b/d39/d6a/d6e/def/db8/f10d 0 2024-03-11T19:56:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/828: fsync d10/d16/d2b/d8d/fdd 0 2024-03-11T19:56:35.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/829: symlink d10/d16/d1f/d51/d9c/l10e 0 2024-03-11T19:56:35.307 INFO:tasks.workunit.client.0.smithi107.stdout:6/830: stat d10/d16/d2b/f43 0 2024-03-11T19:56:35.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/966: creat da/d14/d18/d1e/d9b/d72/dc8/d11c/f136 x:0 0 0 2024-03-11T19:56:35.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/831: creat d10/d16/d1f/d51/d9c/db6/f10f x:0 0 0 2024-03-11T19:56:35.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/967: rmdir da/d14/d18/d1e/d9b/d72 39 2024-03-11T19:56:35.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/809: dwrite d4/d7/d1f/d2b/d24/fd5 [0,4194304] 0 2024-03-11T19:56:35.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/810: link d4/d7/d2f/d33/lc0 d4/d7/d2f/d33/d7b/l100 0 2024-03-11T19:56:35.334 INFO:tasks.workunit.client.0.smithi107.stdout:6/832: write d10/d16/d1f/d51/d76/d7f/fd0 [4394148,117117] 0 2024-03-11T19:56:35.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/968: write da/d14/d38/fef [1937457,80596] 0 2024-03-11T19:56:35.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/833: stat d10/d16/d2b/d39/d6a/fff 0 2024-03-11T19:56:35.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/834: dread - d10/d3d/d54/fd2 zero size 2024-03-11T19:56:35.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/835: write d10/d19/d3b/d70/dbb/fd6 [271832,96502] 0 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:5/969: symlink da/d14/d18/l137 0 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:2/813: sync 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/803: sync 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:1/840: sync 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:9/852: sync 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/804: readlink dd/d13/d18/d81/ddc/le2 0 2024-03-11T19:56:35.336 INFO:tasks.workunit.client.0.smithi107.stdout:7/931: sync 2024-03-11T19:56:35.337 INFO:tasks.workunit.client.0.smithi107.stdout:8/805: read - dd/de/d1b/d4b/fb7 zero size 2024-03-11T19:56:35.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/814: mknod d1/d1a/d34/d6a/d104/c111 0 2024-03-11T19:56:35.338 INFO:tasks.workunit.client.0.smithi107.stdout:7/932: symlink d6/da/d49/dee/l123 0 2024-03-11T19:56:35.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/806: creat dd/de/d1b/d20/d56/d87/ffe x:0 0 0 2024-03-11T19:56:35.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/807: dread - dd/d13/d32/f98 zero size 2024-03-11T19:56:35.339 INFO:tasks.workunit.client.0.smithi107.stdout:7/933: write d6/f7b [179493,25780] 0 2024-03-11T19:56:35.343 INFO:tasks.workunit.client.0.smithi107.stdout:6/836: mknod d10/d16/d1f/c110 0 2024-03-11T19:56:35.343 INFO:tasks.workunit.client.0.smithi107.stdout:6/837: write d10/d16/d1f/f25 [1585812,22917] 0 2024-03-11T19:56:35.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/838: chown d10/d16/d2b/d8d 932 1 2024-03-11T19:56:35.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/841: rmdir d6/d7e/d81 39 2024-03-11T19:56:35.351 INFO:tasks.workunit.client.0.smithi107.stdout:6/839: getdents d10/d19/d3b/d70/d8a 0 2024-03-11T19:56:35.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/842: rename d6/d28/dc4/la1 to d6/d7e/d81/d27/d2c/d32/dc1/dc9/def/l115 0 2024-03-11T19:56:35.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/843: symlink d6/d7e/d81/d27/l116 0 2024-03-11T19:56:35.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/844: chown d6/d7e/d81/d27/dfa/dbb 3 1 2024-03-11T19:56:35.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/845: fsync d6/d7e/d81/f29 0 2024-03-11T19:56:35.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/846: rename d6/d7e/d81/d27/d2c/l51 to d6/d7e/d81/d27/d2c/d7d/l117 0 2024-03-11T19:56:35.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/847: chown d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/c10e 2152123 1 2024-03-11T19:56:35.391 INFO:tasks.workunit.client.0.smithi107.stdout:4/811: dwrite d4/d7/d2f/d33/fee [0,4194304] 0 2024-03-11T19:56:35.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/934: write d6/d9/d57/d65/f74 [1102386,91216] 0 2024-03-11T19:56:35.403 INFO:tasks.workunit.client.0.smithi107.stdout:9/853: dread d4/d9/dae/f77 [0,4194304] 0 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/935: getdents d6/da/d16/d5c/d7d/df0 0 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/936: dread - d6/da/d16/d5c/d7d/df0/f101 zero size 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/937: chown d6/da/d4d/fd6 33 1 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/938: chown d6/d9/d57/d65/c115 31140 1 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/939: readlink d6/da/d16/d5c/d7d/d60/d61/d78/l121 0 2024-03-11T19:56:35.406 INFO:tasks.workunit.client.0.smithi107.stdout:9/854: symlink d4/d1d/d5c/d97/d125/l128 0 2024-03-11T19:56:35.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/855: read d4/d1d/d1f/d28/fa2 [308906,42954] 0 2024-03-11T19:56:35.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/808: dwrite dd/d13/d32/fbe [0,4194304] 0 2024-03-11T19:56:35.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/809: chown dd/d16/f19 49734879 1 2024-03-11T19:56:35.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/810: mknod dd/de/d49/cff 0 2024-03-11T19:56:35.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/811: readlink dd/d13/d32/l6e 0 2024-03-11T19:56:35.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/812: dread - dd/de/d1b/d20/f9e zero size 2024-03-11T19:56:35.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/813: chown dd/de/d1b/d20/d5e/f65 5447809 1 2024-03-11T19:56:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/970: dwrite da/d14/d18/d1e/d9b/d72/fac [0,4194304] 0 2024-03-11T19:56:35.416 INFO:tasks.workunit.client.0.smithi107.stdout:8/814: creat dd/de/d24/d28/d47/f100 x:0 0 0 2024-03-11T19:56:35.418 INFO:tasks.workunit.client.0.smithi107.stdout:8/815: getdents dd/d13/dd7 0 2024-03-11T19:56:35.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/848: write d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/f8f [21623,99926] 0 2024-03-11T19:56:35.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/849: mkdir d6/d118 0 2024-03-11T19:56:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/850: creat d6/d7e/d81/d27/d2c/d72/dc3/f119 x:0 0 0 2024-03-11T19:56:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/851: chown d6/d7e/d81/d27/d2c/d32/d58/fc8 26292874 1 2024-03-11T19:56:35.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/856: rename d4/d1d/d5c/d97/fbd to d4/d1d/d5c/d64/db3/d100/f129 0 2024-03-11T19:56:35.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/857: chown d4/d1d/d4b/c5f 193 1 2024-03-11T19:56:35.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/840: dread d10/d16/d24/d2e/f86 [0,4194304] 0 2024-03-11T19:56:35.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/841: write d10/d16/d1f/d51/d7a/fe1 [1594945,80545] 0 2024-03-11T19:56:35.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/858: truncate d4/d9/dae/fe3 2614195 0 2024-03-11T19:56:35.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/971: rename da/d14/d43/d47/l8e to da/d14/d18/d1e/d124/l138 0 2024-03-11T19:56:35.437 INFO:tasks.workunit.client.0.smithi107.stdout:5/972: read da/d14/d18/f27 [6652396,98877] 0 2024-03-11T19:56:35.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/852: dread d6/d28/f10a [0,4194304] 0 2024-03-11T19:56:35.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/853: write d6/d86/dc0/f102 [266823,85020] 0 2024-03-11T19:56:35.443 INFO:tasks.workunit.client.0.smithi107.stdout:1/854: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/f11a x:0 0 0 2024-03-11T19:56:35.444 INFO:tasks.workunit.client.0.smithi107.stdout:1/855: mkdir d6/d7e/d81/d27/dfa/dbb/d11b 0 2024-03-11T19:56:35.446 INFO:tasks.workunit.client.0.smithi107.stdout:1/856: rename d6/d7e/da5/d10d to d6/d7e/d81/d27/d2c/d32/dc1/dc9/d11c 0 2024-03-11T19:56:35.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/857: dread d6/d7e/d81/d27/d2c/d32/f8d [0,4194304] 0 2024-03-11T19:56:35.462 INFO:tasks.workunit.client.0.smithi107.stdout:1/858: rename d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/dd6/ff9 to d6/d7e/d81/d27/dfa/dff/f11d 0 2024-03-11T19:56:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:1/859: symlink d6/d7e/d81/d27/d2c/d32/dc1/dc9/l11e 0 2024-03-11T19:56:35.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/842: dread d10/d16/f1e [0,4194304] 0 2024-03-11T19:56:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:7/940: read d6/da/d16/d5c/d7d/d60/d61/d78/f102 [2429662,108992] 0 2024-03-11T19:56:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:7/941: truncate d6/d9/d57/f9e 842824 0 2024-03-11T19:56:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/812: dwrite d4/d7/d2f/d3c/ffa [0,4194304] 0 2024-03-11T19:56:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:4/813: unlink d4/d7/d2f/d3b/f58 0 2024-03-11T19:56:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:4/814: dread - d4/d7/d1f/d2b/d19/d75/d8e/fdd zero size 2024-03-11T19:56:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/942: read d6/d9/d57/d65/fc7 [65275,43143] 0 2024-03-11T19:56:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/943: chown d6/da/d16/d91/dc8/d80/ddd 265429 1 2024-03-11T19:56:35.475 INFO:tasks.workunit.client.0.smithi107.stdout:7/944: mknod d6/d21/dab/c124 0 2024-03-11T19:56:35.475 INFO:tasks.workunit.client.0.smithi107.stdout:7/945: write d6/da/d16/d5c/d7d/d60/d61/fb3 [749549,14994] 0 2024-03-11T19:56:35.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/946: mkdir d6/d9/d57/da2/d125 0 2024-03-11T19:56:35.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/843: link d10/d16/d2b/d39/d6a/d6e/def/f69 d10/d16/df6/f111 0 2024-03-11T19:56:35.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/844: chown d10/d16/d1f/d51/d76/l10c 857738 1 2024-03-11T19:56:35.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/845: truncate d10/d16/d1f/f28 3832198 0 2024-03-11T19:56:35.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/846: getdents d10/d16/d2b/d39/d6a/d6e 0 2024-03-11T19:56:35.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/847: creat d10/d16/d2b/f112 x:0 0 0 2024-03-11T19:56:35.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/848: symlink d10/d19/d3b/l113 0 2024-03-11T19:56:35.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/849: rmdir d10/d16/d2b/d39/dca 39 2024-03-11T19:56:35.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/850: fsync d10/d3d/d54/f80 0 2024-03-11T19:56:35.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/851: fsync d10/d19/d3b/ddb/ff3 0 2024-03-11T19:56:35.489 INFO:tasks.workunit.client.0.smithi107.stdout:6/852: creat d10/d3d/d54/f114 x:0 0 0 2024-03-11T19:56:35.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/853: read - d10/d3d/d54/f114 zero size 2024-03-11T19:56:35.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/854: creat d10/d16/d24/f115 x:0 0 0 2024-03-11T19:56:35.491 INFO:tasks.workunit.client.0.smithi107.stdout:8/816: dwrite dd/f58 [0,4194304] 0 2024-03-11T19:56:35.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/815: rename d4/d7/d1f/d2b to d4/d7/d2f/d3b/d101 0 2024-03-11T19:56:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:8/817: symlink dd/dac/l101 0 2024-03-11T19:56:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:8/818: truncate dd/d13/d18/d81/ddc/ff5 1038548 0 2024-03-11T19:56:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/816: read d4/d7/d2f/d3b/d101/d19/fa2 [40654,12464] 0 2024-03-11T19:56:35.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/817: link d4/d7/db/d5e/fcc d4/d7/d1f/f102 0 2024-03-11T19:56:35.500 INFO:tasks.workunit.client.0.smithi107.stdout:8/819: mkdir dd/d13/d32/d51/d102 0 2024-03-11T19:56:35.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/820: write dd/de/d1b/d20/d5e/fc7 [4723723,124455] 0 2024-03-11T19:56:35.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/947: dread d6/d9/d57/d65/fc6 [0,4194304] 0 2024-03-11T19:56:35.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/818: rename d4/d7/d2f/d3b/d101/d24/fd5 to d4/d7/d2f/d3b/d101/d24/f103 0 2024-03-11T19:56:35.512 INFO:tasks.workunit.client.0.smithi107.stdout:8/821: mknod dd/d13/d18/d81/c103 0 2024-03-11T19:56:35.512 INFO:tasks.workunit.client.0.smithi107.stdout:8/822: readlink dd/de/d49/d7e/l8c 0 2024-03-11T19:56:35.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/819: truncate d4/d7/d2f/d33/f66 262806 0 2024-03-11T19:56:35.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/820: getdents d4/d7/d2f/d3b/d101/d19/d75/d8e 0 2024-03-11T19:56:35.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/821: fsync d4/d7/db/d5e/fed 0 2024-03-11T19:56:35.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/822: write d4/d7/da0/fb4 [1506624,52217] 0 2024-03-11T19:56:35.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/823: fsync d4/d7/db/d5c/fc4 0 2024-03-11T19:56:35.516 INFO:tasks.workunit.client.0.smithi107.stdout:4/824: mknod d4/d7/db/dde/c104 0 2024-03-11T19:56:35.516 INFO:tasks.workunit.client.0.smithi107.stdout:4/825: write d4/d7/f8 [2945556,128454] 0 2024-03-11T19:56:35.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/859: dwrite d4/d1d/d5c/fa6 [4194304,4194304] 0 2024-03-11T19:56:35.522 INFO:tasks.workunit.client.0.smithi107.stdout:1/860: dread d6/f1d [4194304,4194304] 0 2024-03-11T19:56:35.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/861: mknod d6/d7e/da5/dc7/c11f 0 2024-03-11T19:56:35.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/860: creat d4/d1d/d1f/d28/f12a x:0 0 0 2024-03-11T19:56:35.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/862: creat d6/d7e/d81/d27/d2c/d32/dc1/f120 x:0 0 0 2024-03-11T19:56:35.531 INFO:tasks.workunit.client.0.smithi107.stdout:1/863: link d6/d7e/d81/l20 d6/d7e/d81/d27/d2c/d32/dc1/dc9/l121 0 2024-03-11T19:56:35.532 INFO:tasks.workunit.client.0.smithi107.stdout:1/864: chown d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/l10f 10 1 2024-03-11T19:56:35.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/823: dread dd/f69 [0,4194304] 0 2024-03-11T19:56:35.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/824: creat dd/de/d49/d46/f104 x:0 0 0 2024-03-11T19:56:35.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/825: chown dd/de/d49/d46/f104 55558051 1 2024-03-11T19:56:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/855: dwrite d10/d16/d24/d2e/d9d/fa8 [4194304,4194304] 0 2024-03-11T19:56:35.544 INFO:tasks.workunit.client.0.smithi107.stdout:6/856: creat d10/d19/d3b/d70/dbb/f116 x:0 0 0 2024-03-11T19:56:35.559 INFO:tasks.workunit.client.0.smithi107.stdout:4/826: dwrite d4/d7/d2f/d3b/d101/f56 [0,4194304] 0 2024-03-11T19:56:35.559 INFO:tasks.workunit.client.0.smithi107.stdout:4/827: dread - d4/d7/db/d5c/f8a zero size 2024-03-11T19:56:35.559 INFO:tasks.workunit.client.0.smithi107.stdout:4/828: fsync d4/d7/d2f/d3b/d101/d19/d75/d8e/fa8 0 2024-03-11T19:56:35.561 INFO:tasks.workunit.client.0.smithi107.stdout:4/829: symlink d4/d7/d2f/d33/d7b/l105 0 2024-03-11T19:56:35.573 INFO:tasks.workunit.client.0.smithi107.stdout:9/861: dwrite d4/d1d/d1f/d28/d2c/d96/fa4 [4194304,4194304] 0 2024-03-11T19:56:35.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/865: dwrite d6/d7e/d81/d27/f3e [0,4194304] 0 2024-03-11T19:56:35.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/826: write dd/de/f48 [230290,78619] 0 2024-03-11T19:56:35.603 INFO:tasks.workunit.client.0.smithi107.stdout:4/830: dwrite d4/d7/d2f/d3b/d101/d93/fe0 [0,4194304] 0 2024-03-11T19:56:35.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/831: readlink d4/d7/d2f/d3b/d101/d24/d26/d29/d36/lf7 0 2024-03-11T19:56:35.605 INFO:tasks.workunit.client.0.smithi107.stdout:4/832: creat d4/d7/d2f/d33/d7b/f106 x:0 0 0 2024-03-11T19:56:35.606 INFO:tasks.workunit.client.0.smithi107.stdout:4/833: creat d4/d7/d2f/d3b/d101/d19/d75/d8e/f107 x:0 0 0 2024-03-11T19:56:35.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/857: dwrite d10/d16/d24/d2e/f86 [0,4194304] 0 2024-03-11T19:56:35.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/858: creat d10/d3d/d54/f117 x:0 0 0 2024-03-11T19:56:35.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/815: sync 2024-03-11T19:56:35.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/816: mknod d1/d1a/d34/d4f/d63/c112 0 2024-03-11T19:56:35.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/817: chown d1/cd 18051 1 2024-03-11T19:56:35.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/827: dwrite dd/de/d1b/f7a [4194304,4194304] 0 2024-03-11T19:56:35.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/828: fsync dd/d13/d18/f1a 0 2024-03-11T19:56:35.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/829: mkdir dd/d13/d32/d51/d105 0 2024-03-11T19:56:35.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/830: write dd/de/d1b/f1d [2306536,31869] 0 2024-03-11T19:56:35.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/831: mknod dd/d13/d18/d81/ddc/de8/c106 0 2024-03-11T19:56:35.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/832: symlink dd/de/d24/d28/de1/l107 0 2024-03-11T19:56:35.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/833: creat dd/d13/f108 x:0 0 0 2024-03-11T19:56:35.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/862: dwrite d4/d1d/d1f/d28/d2c/f62 [0,4194304] 0 2024-03-11T19:56:35.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/863: readlink d4/d1d/d5c/l63 0 2024-03-11T19:56:35.640 INFO:tasks.workunit.client.0.smithi107.stdout:9/864: dread d4/d1d/d1f/d28/d2c/f49 [0,4194304] 0 2024-03-11T19:56:35.641 INFO:tasks.workunit.client.0.smithi107.stdout:9/865: getdents d4/d9/dae 0 2024-03-11T19:56:35.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/866: mkdir d4/d106/d12b 0 2024-03-11T19:56:35.655 INFO:tasks.workunit.client.0.smithi107.stdout:4/834: dwrite d4/d7/d2f/d3c/f77 [0,4194304] 0 2024-03-11T19:56:35.663 INFO:tasks.workunit.client.0.smithi107.stdout:4/835: dread d4/d7/d1f/d82/f83 [0,4194304] 0 2024-03-11T19:56:35.664 INFO:tasks.workunit.client.0.smithi107.stdout:6/859: dwrite d10/d16/d2b/d39/d6a/f83 [4194304,4194304] 0 2024-03-11T19:56:35.675 INFO:tasks.workunit.client.0.smithi107.stdout:2/818: dwrite d1/d38/d75/fb0 [0,4194304] 0 2024-03-11T19:56:35.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/860: write d10/d3d/ff5 [2438074,61432] 0 2024-03-11T19:56:35.679 INFO:tasks.workunit.client.0.smithi107.stdout:6/861: mkdir d10/d16/d2b/da5/d118 0 2024-03-11T19:56:35.682 INFO:tasks.workunit.client.0.smithi107.stdout:6/862: truncate d10/d16/d1f/d51/d76/d7f/dbd/fda 1947753 0 2024-03-11T19:56:35.684 INFO:tasks.workunit.client.0.smithi107.stdout:4/836: write d4/d7/d2f/d3c/f77 [2211767,91157] 0 2024-03-11T19:56:35.684 INFO:tasks.workunit.client.0.smithi107.stdout:4/837: write d4/d7/d2f/d3b/f7e [571343,95535] 0 2024-03-11T19:56:35.693 INFO:tasks.workunit.client.0.smithi107.stdout:8/834: dwrite dd/de/d1b/d20/f29 [4194304,4194304] 0 2024-03-11T19:56:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/835: rmdir dd/de/d24/d28 39 2024-03-11T19:56:35.702 INFO:tasks.workunit.client.0.smithi107.stdout:4/838: read d4/d7/d1f/f52 [284715,2213] 0 2024-03-11T19:56:35.720 INFO:tasks.workunit.client.0.smithi107.stdout:2/819: dwrite d1/d1d/d64/fe3 [0,4194304] 0 2024-03-11T19:56:35.720 INFO:tasks.workunit.client.0.smithi107.stdout:6/863: dwrite d10/d16/d1f/d51/d76/f88 [4194304,4194304] 0 2024-03-11T19:56:35.723 INFO:tasks.workunit.client.0.smithi107.stdout:2/820: mknod d1/d1a/d34/d4f/db3/c113 0 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:2/821: mknod d1/d1a/d1e/c114 0 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:2/822: write d1/d1a/d34/d4f/d63/f6f [1762594,13125] 0 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/973: sync 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:2/823: creat d1/d1a/d34/f115 x:0 0 0 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/974: write da/d14/d18/d1e/f65 [411221,75792] 0 2024-03-11T19:56:35.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/975: mkdir da/d14/d43/d9e/dad/d139 0 2024-03-11T19:56:35.732 INFO:tasks.workunit.client.0.smithi107.stdout:6/864: rmdir d10/d16/d2b/de2 0 2024-03-11T19:56:35.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/839: dread d4/d7/d2f/d33/fee [0,4194304] 0 2024-03-11T19:56:35.734 INFO:tasks.workunit.client.0.smithi107.stdout:4/840: mknod d4/d7/db/c108 0 2024-03-11T19:56:35.734 INFO:tasks.workunit.client.0.smithi107.stdout:6/865: rmdir d10/d16/d2b/d8d 39 2024-03-11T19:56:35.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/824: dread d1/d1a/d8e/fec [0,4194304] 0 2024-03-11T19:56:35.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/825: fdatasync d1/d21/d35/d67/f6e 0 2024-03-11T19:56:35.736 INFO:tasks.workunit.client.0.smithi107.stdout:4/841: creat d4/d7/d2f/d3b/d101/f109 x:0 0 0 2024-03-11T19:56:35.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/826: readlink d1/d38/l105 0 2024-03-11T19:56:35.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/842: creat d4/d7/db/f10a x:0 0 0 2024-03-11T19:56:35.740 INFO:tasks.workunit.client.0.smithi107.stdout:2/827: rename d1/d21/d35/c5f to d1/d38/c116 0 2024-03-11T19:56:35.740 INFO:tasks.workunit.client.0.smithi107.stdout:2/828: chown d1/d1a/d1e/d26/dd6 626 1 2024-03-11T19:56:35.740 INFO:tasks.workunit.client.0.smithi107.stdout:4/843: truncate d4/d7/da0/fe2 3751078 0 2024-03-11T19:56:35.740 INFO:tasks.workunit.client.0.smithi107.stdout:4/844: chown d4/d7/d2f/f68 0 1 2024-03-11T19:56:35.741 INFO:tasks.workunit.client.0.smithi107.stdout:2/829: truncate d1/d21/d35/fc2 989682 0 2024-03-11T19:56:35.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/830: mkdir d1/d38/d117 0 2024-03-11T19:56:35.743 INFO:tasks.workunit.client.0.smithi107.stdout:2/831: write d1/d21/d35/d67/fe8 [907344,115163] 0 2024-03-11T19:56:35.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/832: rmdir d1/d21/dbb 39 2024-03-11T19:56:35.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/833: readlink d1/d1a/d1e/d55/l101 0 2024-03-11T19:56:35.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/834: chown d1/fc 12 1 2024-03-11T19:56:35.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/835: creat d1/d38/d75/d76/f118 x:0 0 0 2024-03-11T19:56:35.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/836: chown d1/d38/d75/c110 457537 1 2024-03-11T19:56:35.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/837: creat d1/d38/d117/f119 x:0 0 0 2024-03-11T19:56:35.776 INFO:tasks.workunit.client.0.smithi107.stdout:2/838: symlink d1/d1d/d64/l11a 0 2024-03-11T19:56:35.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/845: read d4/d7/d2f/d3c/fc6 [2566891,67560] 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/976: dwrite da/d14/d18/dff/d105/d10f/f11e [0,4194304] 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/977: creat da/d14/d43/d9e/f13a x:0 0 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/978: mknod da/d14/d18/d1e/d124/c13b 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/846: dread d4/d7/d2f/d3b/d101/d24/d26/d29/f3f [0,4194304] 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/847: creat d4/d7/d2f/f10b x:0 0 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/848: chown d4/d7/d2f/d3b/d101/d24/d26/d29/cd6 361 1 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/849: fdatasync d4/d7/d2f/d3b/d101/d24/f6a 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/850: fdatasync d4/d7/d2f/d3b/d101/d19/fa7 0 2024-03-11T19:56:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/851: rmdir d4/d7/d2f/d3b/d101/d19/d75/d8e/de9 39 2024-03-11T19:56:35.812 INFO:tasks.workunit.client.0.smithi107.stdout:4/852: dwrite d4/d7/d2f/d3b/d101/d19/d75/d8e/fdd [0,4194304] 0 2024-03-11T19:56:35.852 INFO:tasks.workunit.client.0.smithi107.stdout:4/853: dwrite d4/d7/d2f/d3b/fac [0,4194304] 0 2024-03-11T19:56:35.900 INFO:tasks.workunit.client.0.smithi107.stdout:4/854: dwrite d4/d7/d2f/fe8 [0,4194304] 0 2024-03-11T19:56:35.919 INFO:tasks.workunit.client.0.smithi107.stdout:7/948: sync 2024-03-11T19:56:35.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/855: dwrite d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79/faa [0,4194304] 0 2024-03-11T19:56:35.973 INFO:tasks.workunit.client.0.smithi107.stdout:7/949: dwrite d6/da/d16/d5c/ff9 [0,4194304] 0 2024-03-11T19:56:35.974 INFO:tasks.workunit.client.0.smithi107.stdout:4/856: truncate d4/d7/d2f/d3b/d101/d24/d26/d29/f3f 566272 0 2024-03-11T19:56:35.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/950: mkdir d6/da/d16/d91/dc8/d80/d103/d126 0 2024-03-11T19:56:35.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/951: write d6/da/d16/f3f [3733901,101495] 0 2024-03-11T19:56:35.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/952: read d6/da/fbc [21102,27999] 0 2024-03-11T19:56:35.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/857: unlink d4/d7/d2f/d3b/d101/d19/f23 0 2024-03-11T19:56:35.983 INFO:tasks.workunit.client.0.smithi107.stdout:4/858: symlink d4/d7/d2f/d33/d7b/dca/l10c 0 2024-03-11T19:56:35.986 INFO:tasks.workunit.client.0.smithi107.stdout:4/859: link d4/d7/d1f/l87 d4/d7/db/d5c/l10d 0 2024-03-11T19:56:35.988 INFO:tasks.workunit.client.0.smithi107.stdout:4/860: rmdir d4/d7/d2f/d3b/d101/d19/d9a 39 2024-03-11T19:56:35.989 INFO:tasks.workunit.client.0.smithi107.stdout:4/861: truncate d4/d7/d2f/d3c/f73 693296 0 2024-03-11T19:56:35.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/862: rename d4/d7/d2f/d3b/c6c to d4/d7/d2f/d3b/d101/d93/c10e 0 2024-03-11T19:56:35.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/863: chown d4/d7/d2f/d3b/d101/d93/dc3/dd0/fec 30 1 2024-03-11T19:56:35.993 INFO:tasks.workunit.client.0.smithi107.stdout:1/866: sync 2024-03-11T19:56:35.993 INFO:tasks.workunit.client.0.smithi107.stdout:8/836: sync 2024-03-11T19:56:35.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/867: sync 2024-03-11T19:56:35.993 INFO:tasks.workunit.client.0.smithi107.stdout:2/839: sync 2024-03-11T19:56:35.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/866: sync 2024-03-11T19:56:35.994 INFO:tasks.workunit.client.0.smithi107.stdout:5/979: sync 2024-03-11T19:56:35.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/840: fsync d1/d1d/d64/fe3 0 2024-03-11T19:56:35.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/841: dread - d1/d38/d75/d76/f10c zero size 2024-03-11T19:56:35.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/842: creat d1/d1a/d34/f11b x:0 0 0 2024-03-11T19:56:35.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/864: truncate d4/d7/da0/fe2 3002860 0 2024-03-11T19:56:35.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/843: creat d1/d1a/d1e/d103/d106/f11c x:0 0 0 2024-03-11T19:56:36.000 INFO:tasks.workunit.client.0.smithi107.stdout:9/868: creat d4/d1d/d4b/dfd/f12c x:0 0 0 2024-03-11T19:56:36.000 INFO:tasks.workunit.client.0.smithi107.stdout:8/837: getdents dd/de/d49 0 2024-03-11T19:56:36.000 INFO:tasks.workunit.client.0.smithi107.stdout:8/838: chown dd/d16/f7f 1 1 2024-03-11T19:56:36.000 INFO:tasks.workunit.client.0.smithi107.stdout:8/839: dread - dd/d13/ff1 zero size 2024-03-11T19:56:36.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/980: rmdir da/d14/d18/d1e/d9b/db3 0 2024-03-11T19:56:36.002 INFO:tasks.workunit.client.0.smithi107.stdout:2/844: creat d1/d1a/d34/d4f/d63/daa/f11d x:0 0 0 2024-03-11T19:56:36.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/869: truncate d4/d1d/d1f/d28/d5d/f7f 3441958 0 2024-03-11T19:56:36.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/870: truncate d4/d9/dae/d6d/ff7 477588 0 2024-03-11T19:56:36.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/981: creat da/d14/d43/d7f/f13c x:0 0 0 2024-03-11T19:56:36.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/982: write da/d14/d38/d59/f69 [1931262,40277] 0 2024-03-11T19:56:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/867: unlink d10/d16/d2b/d39/d6a/d6e/def/fa9 0 2024-03-11T19:56:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/868: dread - d10/d19/d3b/d70/dbb/f116 zero size 2024-03-11T19:56:36.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/869: unlink d10/d16/f1e 0 2024-03-11T19:56:36.023 INFO:tasks.workunit.client.0.smithi107.stdout:8/840: write dd/de/f4e [4585394,30577] 0 2024-03-11T19:56:36.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/841: symlink dd/d13/dd7/l109 0 2024-03-11T19:56:36.029 INFO:tasks.workunit.client.0.smithi107.stdout:8/842: creat dd/de/f10a x:0 0 0 2024-03-11T19:56:36.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/871: dread d4/d1c/d37/fbb [0,4194304] 0 2024-03-11T19:56:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/867: dread d6/d7e/d81/d27/d2c/d72/fa6 [4194304,4194304] 0 2024-03-11T19:56:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/868: chown d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3 641194 1 2024-03-11T19:56:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/869: truncate d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/fe1 480664 0 2024-03-11T19:56:36.043 INFO:tasks.workunit.client.0.smithi107.stdout:6/870: rmdir d10/d16/d2b 39 2024-03-11T19:56:36.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/871: dread - d10/fad zero size 2024-03-11T19:56:36.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/872: mknod d4/d1d/d1f/d28/d2c/c12d 0 2024-03-11T19:56:36.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/873: truncate d4/d1d/d5c/d64/db3/fc8 1883295 0 2024-03-11T19:56:36.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/874: creat d4/d1d/d1f/d28/d5d/df2/ddb/f12e x:0 0 0 2024-03-11T19:56:36.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/875: fdatasync d4/d1d/d1f/d28/d2c/f49 0 2024-03-11T19:56:36.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/876: symlink d4/d1d/d5c/d97/ddc/l12f 0 2024-03-11T19:56:36.052 INFO:tasks.workunit.client.0.smithi107.stdout:2/845: write d1/d1a/d34/d4f/db3/fb7 [3298476,5558] 0 2024-03-11T19:56:36.052 INFO:tasks.workunit.client.0.smithi107.stdout:2/846: chown d1/d1a/d1e/d41/f51 180126971 1 2024-03-11T19:56:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:7/953: dwrite d6/da/d16/d5c/d7d/d1b/dfd/fb2 [4194304,4194304] 0 2024-03-11T19:56:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:7/954: stat d6/da/d16/d5c/d7d/d60/d61/d78/f8d 0 2024-03-11T19:56:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:7/955: write d6/da/d4d/fd6 [620373,106436] 0 2024-03-11T19:56:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:9/877: symlink d4/d9/dae/d6d/l130 0 2024-03-11T19:56:36.054 INFO:tasks.workunit.client.0.smithi107.stdout:7/956: creat d6/da/d4d/f127 x:0 0 0 2024-03-11T19:56:36.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/878: truncate d4/d1d/d1f/d28/f67 110988 0 2024-03-11T19:56:36.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/879: chown d4/d1c/d37/f8b 752814 1 2024-03-11T19:56:36.056 INFO:tasks.workunit.client.0.smithi107.stdout:7/957: symlink d6/da/d49/dee/d104/l128 0 2024-03-11T19:56:36.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/843: dwrite dd/de/d1b/f9a [0,4194304] 0 2024-03-11T19:56:36.077 INFO:tasks.workunit.client.0.smithi107.stdout:5/983: dwrite da/f86 [0,4194304] 0 2024-03-11T19:56:36.081 INFO:tasks.workunit.client.0.smithi107.stdout:1/870: dread d6/f48 [0,4194304] 0 2024-03-11T19:56:36.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/871: mknod d6/d86/c122 0 2024-03-11T19:56:36.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/872: stat d6/d7e/da5 0 2024-03-11T19:56:36.107 INFO:tasks.workunit.client.0.smithi107.stdout:2/847: dwrite d1/fe2 [0,4194304] 0 2024-03-11T19:56:36.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/872: dwrite d10/d16/d24/d2e/d9d/f92 [0,4194304] 0 2024-03-11T19:56:36.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/984: dwrite da/d14/d96/dc1/f103 [0,4194304] 0 2024-03-11T19:56:36.130 INFO:tasks.workunit.client.0.smithi107.stdout:5/985: chown da/d14/d38/d59/c29 1265 1 2024-03-11T19:56:36.133 INFO:tasks.workunit.client.0.smithi107.stdout:5/986: mkdir da/d14/d18/dff/d105/d10f/d13d 0 2024-03-11T19:56:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/873: dwrite d6/f10 [0,4194304] 0 2024-03-11T19:56:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/874: write d6/d7e/da5/dc7/feb [659681,29737] 0 2024-03-11T19:56:36.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/875: truncate d6/d86/fbc 2206688 0 2024-03-11T19:56:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/876: creat d6/d7e/da5/f123 x:0 0 0 2024-03-11T19:56:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/877: chown d6/d5b/f78 1262071686 1 2024-03-11T19:56:36.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/958: dwrite d6/d9/dae/fb8 [4194304,4194304] 0 2024-03-11T19:56:36.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/959: chown d6/da/d49/ld8 2984 1 2024-03-11T19:56:36.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/960: read d6/da/f8a [3502199,15993] 0 2024-03-11T19:56:36.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/961: getdents d6/d9/d57/da2/d125 0 2024-03-11T19:56:36.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/844: dwrite dd/d13/ff1 [0,4194304] 0 2024-03-11T19:56:36.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/848: dwrite d1/d1a/d1e/f9b [0,4194304] 0 2024-03-11T19:56:36.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/880: dread d4/d9/dae/f77 [0,4194304] 0 2024-03-11T19:56:36.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/849: symlink d1/d38/d75/dd7/l11e 0 2024-03-11T19:56:36.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/850: fdatasync d1/d1a/d1e/d26/fb2 0 2024-03-11T19:56:36.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/851: dread - d1/d1a/d34/d4f/d63/d100/f107 zero size 2024-03-11T19:56:36.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/852: creat d1/d1a/d1e/d103/f11f x:0 0 0 2024-03-11T19:56:36.160 INFO:tasks.workunit.client.0.smithi107.stdout:6/873: write d10/d16/d2b/d8d/fdd [4133352,78573] 0 2024-03-11T19:56:36.163 INFO:tasks.workunit.client.0.smithi107.stdout:6/874: mknod d10/d16/d1f/d51/d9c/db6/d102/c119 0 2024-03-11T19:56:36.169 INFO:tasks.workunit.client.0.smithi107.stdout:4/865: sync 2024-03-11T19:56:36.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/853: dread d1/d38/d75/fa9 [0,4194304] 0 2024-03-11T19:56:36.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/854: truncate d1/d1d/d64/d97/fab 1351149 0 2024-03-11T19:56:36.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/987: dread da/d14/d18/ff0 [0,4194304] 0 2024-03-11T19:56:36.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/866: symlink d4/d7/d2f/d3b/d101/d24/d26/l10f 0 2024-03-11T19:56:36.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/867: creat d4/d7/d2f/d3b/d101/d24/f110 x:0 0 0 2024-03-11T19:56:36.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/868: chown d4/d7/d2f/d3b/d101/f56 353 1 2024-03-11T19:56:36.203 INFO:tasks.workunit.client.0.smithi107.stdout:7/962: dwrite d6/d9/f85 [0,4194304] 0 2024-03-11T19:56:36.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/878: dwrite d6/d5b/f108 [0,4194304] 0 2024-03-11T19:56:36.207 INFO:tasks.workunit.client.0.smithi107.stdout:1/879: symlink d6/d7e/d81/d27/d2c/d72/dc3/dd4/l124 0 2024-03-11T19:56:36.211 INFO:tasks.workunit.client.0.smithi107.stdout:7/963: rename d6/da/d16/d5c/d7d/d60/d61/fb3 to d6/d21/dab/f129 0 2024-03-11T19:56:36.215 INFO:tasks.workunit.client.0.smithi107.stdout:9/881: dwrite d4/d1d/d1f/d28/f6e [0,4194304] 0 2024-03-11T19:56:36.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/964: creat d6/d9/d57/d65/d10d/f12a x:0 0 0 2024-03-11T19:56:36.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/965: write d6/d9/f112 [1165270,53991] 0 2024-03-11T19:56:36.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/966: creat d6/da/d4d/f12b x:0 0 0 2024-03-11T19:56:36.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/988: dread da/d14/d18/ff0 [0,4194304] 0 2024-03-11T19:56:36.216 INFO:tasks.workunit.client.0.smithi107.stdout:8/845: dwrite dd/de/d24/d28/fcd [0,4194304] 0 2024-03-11T19:56:36.219 INFO:tasks.workunit.client.0.smithi107.stdout:6/875: dwrite d10/d16/d24/d2e/d9d/fa8 [4194304,4194304] 0 2024-03-11T19:56:36.220 INFO:tasks.workunit.client.0.smithi107.stdout:5/989: link da/d14/d43/f6c da/d14/d18/d1e/d124/f13e 0 2024-03-11T19:56:36.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/990: chown da/d14/d18/d44/f10c 0 1 2024-03-11T19:56:36.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/882: dread d4/d1d/d4b/f59 [0,4194304] 0 2024-03-11T19:56:36.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/991: mknod da/d14/d18/d1e/c13f 0 2024-03-11T19:56:36.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/992: creat da/d14/d18/d6d/f140 x:0 0 0 2024-03-11T19:56:36.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/993: dread - da/d14/d18/f106 zero size 2024-03-11T19:56:36.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/855: write d1/f2b [3233361,18961] 0 2024-03-11T19:56:36.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/856: creat d1/d1a/d8e/f120 x:0 0 0 2024-03-11T19:56:36.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/857: dread - d1/d1a/d34/f11b zero size 2024-03-11T19:56:36.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/858: creat d1/d1a/d1e/d103/f121 x:0 0 0 2024-03-11T19:56:36.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/859: read d1/f40 [4170300,35713] 0 2024-03-11T19:56:36.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/860: truncate d1/d1a/d1e/d26/f7a 468822 0 2024-03-11T19:56:36.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/861: write d1/d1a/d34/d4f/d63/d100/f107 [976080,89982] 0 2024-03-11T19:56:36.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/862: truncate d1/d1a/d1e/f9b 5083448 0 2024-03-11T19:56:36.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/863: symlink d1/d1a/d1e/d26/dd6/l122 0 2024-03-11T19:56:36.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/846: link dd/de/d1b/d20/d5e/f80 dd/de/f10b 0 2024-03-11T19:56:36.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/883: dread d4/d1d/d1f/d78/f90 [0,4194304] 0 2024-03-11T19:56:36.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/847: write dd/d13/d18/d81/ddc/fd1 [2740908,97076] 0 2024-03-11T19:56:36.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/848: readlink dd/d13/d18/d81/ddc/le2 0 2024-03-11T19:56:36.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/849: chown dd/de/d24/d28/d47 141 1 2024-03-11T19:56:36.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/876: dwrite d10/f71 [0,4194304] 0 2024-03-11T19:56:36.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/880: dwrite d6/d7e/d81/d27/d2c/d7d/dd8/ff1 [0,4194304] 0 2024-03-11T19:56:36.277 INFO:tasks.workunit.client.0.smithi107.stdout:7/967: dwrite d6/d9/d57/d65/fc7 [0,4194304] 0 2024-03-11T19:56:36.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/881: symlink d6/l125 0 2024-03-11T19:56:36.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/882: fdatasync d6/d7e/da5/dc7/feb 0 2024-03-11T19:56:36.281 INFO:tasks.workunit.client.0.smithi107.stdout:2/864: write d1/d21/d35/fa5 [496754,34294] 0 2024-03-11T19:56:36.282 INFO:tasks.workunit.client.0.smithi107.stdout:7/968: mknod d6/da/d16/d5c/d7d/d1b/dfd/c12c 0 2024-03-11T19:56:36.282 INFO:tasks.workunit.client.0.smithi107.stdout:7/969: chown d6/da/d16/d5c/d7d/d1b/f1c 4680 1 2024-03-11T19:56:36.283 INFO:tasks.workunit.client.0.smithi107.stdout:2/865: mknod d1/d1a/d34/d6a/d104/c123 0 2024-03-11T19:56:36.284 INFO:tasks.workunit.client.0.smithi107.stdout:7/970: rmdir d6/da/d16/d5c/d7d/d60/d61/db9 39 2024-03-11T19:56:36.287 INFO:tasks.workunit.client.0.smithi107.stdout:7/971: mkdir d6/d9/d57/d65/d10d/d12d 0 2024-03-11T19:56:36.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/994: dwrite da/d14/d43/dab/fee [0,4194304] 0 2024-03-11T19:56:36.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/995: write da/d14/d18/d1e/d9b/d73/f8a [4583063,93600] 0 2024-03-11T19:56:36.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/996: stat da/d31/d42 0 2024-03-11T19:56:36.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/997: write da/d14/d96/f88 [4338461,106004] 0 2024-03-11T19:56:36.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/998: dread - da/d14/d18/fdf zero size 2024-03-11T19:56:36.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/999: creat da/d14/d38/d46/dd9/f141 x:0 0 0 2024-03-11T19:56:36.313 INFO:tasks.workunit.client.0.smithi107.stdout:7/972: dread d6/d9/f6d [4194304,4194304] 0 2024-03-11T19:56:36.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/973: rename d6/d9/d57/d65/c115 to d6/d9/d57/da2/c12e 0 2024-03-11T19:56:36.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/974: write d6/d9/d57/d65/f99 [4477365,163] 0 2024-03-11T19:56:36.322 INFO:tasks.workunit.client.0.smithi107.stdout:7/975: rename d6/f2f to d6/da/d16/d91/f12f 0 2024-03-11T19:56:36.322 INFO:tasks.workunit.client.0.smithi107.stdout:7/976: stat d6/da/d16/cf6 0 2024-03-11T19:56:36.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/850: dread dd/d13/d32/d51/fb3 [0,4194304] 0 2024-03-11T19:56:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:9/884: dwrite d4/fcb [0,4194304] 0 2024-03-11T19:56:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/851: dread - dd/de/d24/d28/d47/f100 zero size 2024-03-11T19:56:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/877: dwrite d10/dee/fe6 [0,4194304] 0 2024-03-11T19:56:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/878: write d10/d16/d1f/d51/d76/d7f/dbd/fda [2008182,66673] 0 2024-03-11T19:56:36.338 INFO:tasks.workunit.client.0.smithi107.stdout:2/866: dwrite d1/f15 [0,4194304] 0 2024-03-11T19:56:36.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/879: chown d10/d16/d2b/d39/d6a/d6e/def/db1 7 1 2024-03-11T19:56:36.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/880: truncate fc 1751858 0 2024-03-11T19:56:36.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/881: chown d10/d16/d1f/d51/d76/l10c 6310 1 2024-03-11T19:56:36.343 INFO:tasks.workunit.client.0.smithi107.stdout:6/882: mkdir d10/d16/d2b/da5/d118/d11a 0 2024-03-11T19:56:36.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/883: write d10/d16/d2b/d39/d6a/d6e/def/db8/fcc [5042794,111543] 0 2024-03-11T19:56:36.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/885: creat d4/d9/d9f/dd8/f131 x:0 0 0 2024-03-11T19:56:36.346 INFO:tasks.workunit.client.0.smithi107.stdout:9/886: creat d4/d1d/d1f/d78/d118/f132 x:0 0 0 2024-03-11T19:56:36.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/852: dread dd/d13/ff1 [0,4194304] 0 2024-03-11T19:56:36.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/977: dread d6/d9/f6d [0,4194304] 0 2024-03-11T19:56:36.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/978: unlink d6/d21/c82 0 2024-03-11T19:56:36.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/853: rename dd/d13/d32/d51/c66 to dd/dac/c10c 0 2024-03-11T19:56:36.368 INFO:tasks.workunit.client.0.smithi107.stdout:8/854: write dd/de/d1b/d20/f6a [156867,36840] 0 2024-03-11T19:56:36.368 INFO:tasks.workunit.client.0.smithi107.stdout:8/855: chown dd/d13/d32/cf0 36015182 1 2024-03-11T19:56:36.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/979: dread d6/d9/d57/d65/fef [0,4194304] 0 2024-03-11T19:56:36.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/867: dwrite d1/d21/d35/f8d [0,4194304] 0 2024-03-11T19:56:36.386 INFO:tasks.workunit.client.0.smithi107.stdout:2/868: rename d1/d1a/d1e/fc3 to d1/d1a/d1e/d103/d106/f124 0 2024-03-11T19:56:36.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/887: dwrite d4/d9/d9f/dd8/f131 [0,4194304] 0 2024-03-11T19:56:36.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/869: mkdir d1/d21/d35/dfc/d125 0 2024-03-11T19:56:36.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/870: truncate d1/d1d/d64/d97/dd1/ff3 885993 0 2024-03-11T19:56:36.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/871: chown d1/d38/d75 19852 1 2024-03-11T19:56:36.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/884: dwrite d10/d16/d1f/d51/d9c/f104 [0,4194304] 0 2024-03-11T19:56:36.392 INFO:tasks.workunit.client.0.smithi107.stdout:2/872: truncate d1/d21/dbb/fd0 3104243 0 2024-03-11T19:56:36.393 INFO:tasks.workunit.client.0.smithi107.stdout:6/885: read d10/d16/d2b/d39/d6a/d6e/def/f69 [121937,90604] 0 2024-03-11T19:56:36.397 INFO:tasks.workunit.client.0.smithi107.stdout:2/873: rmdir d1/d38/d117 39 2024-03-11T19:56:36.397 INFO:tasks.workunit.client.0.smithi107.stdout:8/856: dread dd/de/d1b/d20/f43 [0,4194304] 0 2024-03-11T19:56:36.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/874: symlink d1/d1d/d80/l126 0 2024-03-11T19:56:36.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/875: dread - d1/d1a/d34/d4f/d63/db6/fbd zero size 2024-03-11T19:56:36.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/876: chown d1/d1a/d34/d4f/l93 27 1 2024-03-11T19:56:36.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/886: getdents d10/d19/d3b/d70/dbb 0 2024-03-11T19:56:36.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/887: stat d10/d19/d3b/ddb/ff3 0 2024-03-11T19:56:36.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/888: creat d10/d3d/d54/f11b x:0 0 0 2024-03-11T19:56:36.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/889: unlink d10/d16/d2b/d39/dca/fea 0 2024-03-11T19:56:36.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/890: dread - d10/d19/d3b/d70/dbb/f116 zero size 2024-03-11T19:56:36.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/891: dread - d10/d16/d2b/da5/fc4 zero size 2024-03-11T19:56:36.408 INFO:tasks.workunit.client.0.smithi107.stdout:6/892: truncate d10/d16/d1f/f79 2902769 0 2024-03-11T19:56:36.412 INFO:tasks.workunit.client.0.smithi107.stdout:9/888: write d4/d1d/d1f/d28/d2c/f34 [1307454,33583] 0 2024-03-11T19:56:36.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/980: write d6/fd4 [447242,123413] 0 2024-03-11T19:56:36.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/981: fdatasync d6/d9/d57/f9e 0 2024-03-11T19:56:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/869: sync 2024-03-11T19:56:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/883: sync 2024-03-11T19:56:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/884: write d6/d7e/f8b [927705,34394] 0 2024-03-11T19:56:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/885: stat d6/d7e/d81/d27/f7b 0 2024-03-11T19:56:36.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/870: rename d4/d7/ce4 to d4/d7/d2f/d3b/d101/d19/c111 0 2024-03-11T19:56:36.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/893: write d10/d16/d2b/d39/d6a/fd1 [1591480,54269] 0 2024-03-11T19:56:36.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/894: write d10/d16/d24/fd4 [50791,60221] 0 2024-03-11T19:56:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:7/982: read d6/da/d16/d5c/d7d/f5e [4026996,71199] 0 2024-03-11T19:56:36.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/895: dread d10/d16/d1f/f28 [0,4194304] 0 2024-03-11T19:56:36.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/983: dwrite d6/da/d16/d5c/d7d/d1b/f10e [0,4194304] 0 2024-03-11T19:56:36.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/896: dwrite d10/d19/d3b/d70/dbb/f116 [0,4194304] 0 2024-03-11T19:56:36.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/897: mkdir d10/d19/d11c 0 2024-03-11T19:56:36.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/898: chown d10/d16/d24/f5b 20785 1 2024-03-11T19:56:36.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/886: dwrite d6/d5b/fd5 [0,4194304] 0 2024-03-11T19:56:36.509 INFO:tasks.workunit.client.0.smithi107.stdout:4/871: dread d4/d7/d2f/d3b/d101/d19/d9a/fcd [0,4194304] 0 2024-03-11T19:56:36.509 INFO:tasks.workunit.client.0.smithi107.stdout:4/872: truncate d4/d7/d2f/d3b/d101/d19/d75/d8e/de9/ffc 4578016 0 2024-03-11T19:56:36.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/899: creat d10/d16/d2b/d39/d6a/d6e/def/f11d x:0 0 0 2024-03-11T19:56:36.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/900: chown d10/d16/d2b/d39/l81 1 1 2024-03-11T19:56:36.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/901: chown d10/d19/d3b/d70/lbe 3584 1 2024-03-11T19:56:36.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/887: symlink d6/d7e/d81/d27/d2c/d32/dc1/l126 0 2024-03-11T19:56:36.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/857: sync 2024-03-11T19:56:36.515 INFO:tasks.workunit.client.0.smithi107.stdout:1/888: rmdir d6/d7e/d81/d27/d2c/d32 39 2024-03-11T19:56:36.515 INFO:tasks.workunit.client.0.smithi107.stdout:1/889: write d6/d28/f10a [679429,54462] 0 2024-03-11T19:56:36.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/858: write dd/de/d49/d7e/fcb [458680,33896] 0 2024-03-11T19:56:36.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/890: creat d6/d7e/da5/dd9/f127 x:0 0 0 2024-03-11T19:56:36.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/859: rmdir dd/de/d1b/d20/d56/d87 39 2024-03-11T19:56:36.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/860: readlink dd/d16/ldd 0 2024-03-11T19:56:36.522 INFO:tasks.workunit.client.0.smithi107.stdout:4/873: dread d4/d7/d2f/d3b/d101/d19/fa2 [0,4194304] 0 2024-03-11T19:56:36.522 INFO:tasks.workunit.client.0.smithi107.stdout:4/874: write d4/d7/db/f10a [579281,75525] 0 2024-03-11T19:56:36.536 INFO:tasks.workunit.client.0.smithi107.stdout:8/861: dread dd/de/d1b/d20/f6a [0,4194304] 0 2024-03-11T19:56:36.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/984: dwrite d6/d9/f85 [0,4194304] 0 2024-03-11T19:56:36.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/985: creat d6/da/d16/d5c/d7d/df0/f130 x:0 0 0 2024-03-11T19:56:36.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/986: chown d6/da/d16/d5c/d7d/f11d 2354225 1 2024-03-11T19:56:36.542 INFO:tasks.workunit.client.0.smithi107.stdout:7/987: truncate d6/da/d16/d5c/d7d/d60/d61/f10a 88520 0 2024-03-11T19:56:36.542 INFO:tasks.workunit.client.0.smithi107.stdout:7/988: fdatasync d6/da/d16/d5c/d7d/f5e 0 2024-03-11T19:56:36.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/877: sync 2024-03-11T19:56:36.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/878: chown d1/d1a/d1e/d55/ccd 2036 1 2024-03-11T19:56:36.551 INFO:tasks.workunit.client.0.smithi107.stdout:9/889: sync 2024-03-11T19:56:36.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/875: dread d4/d7/f8 [0,4194304] 0 2024-03-11T19:56:36.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/876: write d4/fbe [4660294,9579] 0 2024-03-11T19:56:36.555 INFO:tasks.workunit.client.0.smithi107.stdout:2/879: rmdir d1/d1a/d1e/d41 39 2024-03-11T19:56:36.556 INFO:tasks.workunit.client.0.smithi107.stdout:4/877: rename d4/d7/d2f/d3b/d101/l89 to d4/d7/db/l112 0 2024-03-11T19:56:36.558 INFO:tasks.workunit.client.0.smithi107.stdout:2/880: truncate d1/f15 431903 0 2024-03-11T19:56:36.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/881: fsync d1/d1d/f5b 0 2024-03-11T19:56:36.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/882: creat d1/d38/d75/d76/f127 x:0 0 0 2024-03-11T19:56:36.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/883: truncate d1/d1a/f60 1481241 0 2024-03-11T19:56:36.559 INFO:tasks.workunit.client.0.smithi107.stdout:4/878: rmdir d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79 39 2024-03-11T19:56:36.564 INFO:tasks.workunit.client.0.smithi107.stdout:4/879: creat d4/d7/d2f/d3b/d101/d19/d75/d8e/de9/f113 x:0 0 0 2024-03-11T19:56:36.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/880: truncate d4/d7/d2f/d3b/d101/d19/d75/d8e/fa8 945824 0 2024-03-11T19:56:36.566 INFO:tasks.workunit.client.0.smithi107.stdout:4/881: mknod d4/d7/d2f/d33/d7b/dca/c114 0 2024-03-11T19:56:36.566 INFO:tasks.workunit.client.0.smithi107.stdout:4/882: stat d4/d7/d2f/d3b/d101/d24/d26/d29/l46 0 2024-03-11T19:56:36.571 INFO:tasks.workunit.client.0.smithi107.stdout:6/902: dwrite d10/d16/d24/d2e/d9d/fa8 [0,4194304] 0 2024-03-11T19:56:36.571 INFO:tasks.workunit.client.0.smithi107.stdout:6/903: chown d10/d19/d38 26486103 1 2024-03-11T19:56:36.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/904: creat d10/d16/d2b/d8d/f11e x:0 0 0 2024-03-11T19:56:36.576 INFO:tasks.workunit.client.0.smithi107.stdout:6/905: mknod d10/d16/d1f/d51/d9c/db6/d102/c11f 0 2024-03-11T19:56:36.577 INFO:tasks.workunit.client.0.smithi107.stdout:6/906: truncate d10/d19/d3b/f73 807610 0 2024-03-11T19:56:36.579 INFO:tasks.workunit.client.0.smithi107.stdout:2/884: dread d1/d1a/f7c [0,4194304] 0 2024-03-11T19:56:36.579 INFO:tasks.workunit.client.0.smithi107.stdout:2/885: symlink d1/d38/l128 0 2024-03-11T19:56:36.580 INFO:tasks.workunit.client.0.smithi107.stdout:6/907: link d10/d16/d2b/d39/dca/fd9 d10/d16/d2b/d39/d6a/f120 0 2024-03-11T19:56:36.580 INFO:tasks.workunit.client.0.smithi107.stdout:9/890: dread d4/f53 [0,4194304] 0 2024-03-11T19:56:36.581 INFO:tasks.workunit.client.0.smithi107.stdout:8/862: dwrite dd/de/d49/d46/f104 [0,4194304] 0 2024-03-11T19:56:36.581 INFO:tasks.workunit.client.0.smithi107.stdout:2/886: link d1/d1a/d34/d4f/lb1 d1/d1d/l129 0 2024-03-11T19:56:36.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/887: write d1/d1d/d64/d97/ff7 [547185,6085] 0 2024-03-11T19:56:36.582 INFO:tasks.workunit.client.0.smithi107.stdout:9/891: creat d4/d1d/d5c/d64/d104/f133 x:0 0 0 2024-03-11T19:56:36.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/863: mkdir dd/de/d49/d46/d10d 0 2024-03-11T19:56:36.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/864: write dd/d13/d18/d81/ddc/fe6 [470435,93153] 0 2024-03-11T19:56:36.584 INFO:tasks.workunit.client.0.smithi107.stdout:6/908: creat d10/d16/d2b/da5/d118/f121 x:0 0 0 2024-03-11T19:56:36.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/892: rename d4/d1c/d47/ce6 to d4/d106/c134 0 2024-03-11T19:56:36.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/893: write d4/d1d/d1f/d28/d2c/f34 [1226613,45741] 0 2024-03-11T19:56:36.586 INFO:tasks.workunit.client.0.smithi107.stdout:8/865: creat dd/d13/d32/d51/d105/f10e x:0 0 0 2024-03-11T19:56:36.589 INFO:tasks.workunit.client.0.smithi107.stdout:6/909: truncate d10/d16/d1f/d51/d76/d7f/fd0 2940204 0 2024-03-11T19:56:36.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/888: dread d1/d1a/d1e/d55/fca [0,4194304] 0 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/889: rmdir d1/d21/d35/d67 39 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/890: fdatasync d1/d38/d75/fa0 0 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/891: chown d1/d1a/f60 26076 1 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/892: fsync d1/d1a/d34/f11b 0 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/893: chown d1/d1d/l72 2035 1 2024-03-11T19:56:36.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/894: stat d1/d21/d35/f8d 0 2024-03-11T19:56:36.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/866: read dd/fb4 [2289177,16139] 0 2024-03-11T19:56:36.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/895: write d1/d21/f28 [3494198,11359] 0 2024-03-11T19:56:36.617 INFO:tasks.workunit.client.0.smithi107.stdout:2/896: dread - d1/d1a/d1e/d41/ffb zero size 2024-03-11T19:56:36.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/897: read d1/d1a/d1e/d41/f51 [216350,35765] 0 2024-03-11T19:56:36.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/867: write dd/de/d1b/d20/f43 [186637,55448] 0 2024-03-11T19:56:36.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/868: write dd/d13/d32/fd3 [822099,112923] 0 2024-03-11T19:56:36.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/869: rename dd/de/d49 to dd/de/dfc/d10f 0 2024-03-11T19:56:36.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/870: write dd/de/d1b/d4b/f82 [4465123,100317] 0 2024-03-11T19:56:36.673 INFO:tasks.workunit.client.0.smithi107.stdout:2/898: dwrite d1/d1d/d64/d97/dd1/ff3 [0,4194304] 0 2024-03-11T19:56:36.673 INFO:tasks.workunit.client.0.smithi107.stdout:2/899: stat d1/d1a/d34/d4f/d63/daa/ce7 0 2024-03-11T19:56:36.676 INFO:tasks.workunit.client.0.smithi107.stdout:2/900: rename d1/d1a/d34/d4f/d63/daa/cb8 to d1/d1a/d34/d4f/d63/daa/c12a 0 2024-03-11T19:56:36.677 INFO:tasks.workunit.client.0.smithi107.stdout:2/901: mkdir d1/d1a/d1e/d26/dd6/df4/d12b 0 2024-03-11T19:56:36.680 INFO:tasks.workunit.client.0.smithi107.stdout:2/902: rename d1/d38/lfd to d1/d1a/d1e/d26/dd6/l12c 0 2024-03-11T19:56:36.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/903: mkdir d1/d1d/d64/d12d 0 2024-03-11T19:56:36.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/904: truncate d1/d1a/f2f 2902896 0 2024-03-11T19:56:36.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/905: chown d1/d1a/d34/d4f/c69 553 1 2024-03-11T19:56:36.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/906: mknod d1/c12e 0 2024-03-11T19:56:36.689 INFO:tasks.workunit.client.0.smithi107.stdout:8/871: dwrite dd/de/d1b/f83 [0,4194304] 0 2024-03-11T19:56:36.690 INFO:tasks.workunit.client.0.smithi107.stdout:8/872: creat dd/dac/f110 x:0 0 0 2024-03-11T19:56:36.726 INFO:tasks.workunit.client.0.smithi107.stdout:2/907: dwrite d1/d38/d75/fa0 [0,4194304] 0 2024-03-11T19:56:36.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/873: dwrite dd/de/d1b/d20/d56/d87/f8f [4194304,4194304] 0 2024-03-11T19:56:36.765 INFO:tasks.workunit.client.0.smithi107.stdout:2/908: dwrite d1/d1d/d64/d97/fab [0,4194304] 0 2024-03-11T19:56:36.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/909: creat d1/d21/dbb/f12f x:0 0 0 2024-03-11T19:56:36.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/989: sync 2024-03-11T19:56:36.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/891: sync 2024-03-11T19:56:36.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/990: write d6/da/d16/d91/dc8/d80/fb0 [1285642,102647] 0 2024-03-11T19:56:36.774 INFO:tasks.workunit.client.0.smithi107.stdout:1/892: mkdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c/d128 0 2024-03-11T19:56:36.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/991: mkdir d6/da/d16/d5c/d7d/d60/dd2/d131 0 2024-03-11T19:56:36.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/893: symlink d6/d7e/d81/d27/d2c/d32/dc1/dc9/d11c/l129 0 2024-03-11T19:56:36.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/894: fdatasync d6/d7e/d81/d27/d2c/d32/dc1/f120 0 2024-03-11T19:56:36.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/895: getdents d6/d7e/da5/dd9 0 2024-03-11T19:56:36.779 INFO:tasks.workunit.client.0.smithi107.stdout:7/992: rmdir d6/da/d16/d5c/d7d/d60/d61 39 2024-03-11T19:56:36.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/874: dwrite dd/f99 [0,4194304] 0 2024-03-11T19:56:36.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/875: chown dd/de/dfc/d10f/d46/ca6 422974535 1 2024-03-11T19:56:36.780 INFO:tasks.workunit.client.0.smithi107.stdout:8/876: mkdir dd/de/d1b/d20/d56/da4/dee/d111 0 2024-03-11T19:56:36.780 INFO:tasks.workunit.client.0.smithi107.stdout:8/877: stat dd/de/d1b/d20/d56/da4/cab 0 2024-03-11T19:56:36.782 INFO:tasks.workunit.client.0.smithi107.stdout:7/993: read d6/d9/f112 [67557,90156] 0 2024-03-11T19:56:36.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/878: unlink dd/d13/d32/f98 0 2024-03-11T19:56:36.783 INFO:tasks.workunit.client.0.smithi107.stdout:7/994: creat d6/da/d16/d5c/d7d/d60/d61/d78/f132 x:0 0 0 2024-03-11T19:56:36.783 INFO:tasks.workunit.client.0.smithi107.stdout:7/995: fsync d6/da/d4d/fd5 0 2024-03-11T19:56:36.783 INFO:tasks.workunit.client.0.smithi107.stdout:7/996: dread - d6/da/d16/d5c/d7d/d60/ff2 zero size 2024-03-11T19:56:36.784 INFO:tasks.workunit.client.0.smithi107.stdout:7/997: unlink d6/da/d16/d5c/d7d/d60/ff2 0 2024-03-11T19:56:36.785 INFO:tasks.workunit.client.0.smithi107.stdout:7/998: mknod d6/d9/d57/da2/d125/c133 0 2024-03-11T19:56:36.785 INFO:tasks.workunit.client.0.smithi107.stdout:7/999: write d6/da/d16/f6f [4318072,24939] 0 2024-03-11T19:56:36.799 INFO:tasks.workunit.client.0.smithi107.stdout:1/896: dread d6/f15 [0,4194304] 0 2024-03-11T19:56:36.800 INFO:tasks.workunit.client.0.smithi107.stdout:1/897: stat d6/le 0 2024-03-11T19:56:36.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/898: rmdir d6/d7e/d81/d27/d2c/d32/dc1/dc9/dfe 39 2024-03-11T19:56:36.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/899: chown d6/d7e/d81/d27/d2c/d32/dc1/df7 1682 1 2024-03-11T19:56:36.805 INFO:tasks.workunit.client.0.smithi107.stdout:1/900: symlink d6/d7e/l12a 0 2024-03-11T19:56:36.805 INFO:tasks.workunit.client.0.smithi107.stdout:1/901: mknod d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c/c12b 0 2024-03-11T19:56:36.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/902: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c/d128/f12c x:0 0 0 2024-03-11T19:56:36.807 INFO:tasks.workunit.client.0.smithi107.stdout:1/903: rmdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c 39 2024-03-11T19:56:36.807 INFO:tasks.workunit.client.0.smithi107.stdout:1/904: stat d6/d7e/d81/d27/d2c/d32/dc1/dc9/cde 0 2024-03-11T19:56:36.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/905: fsync d6/d5b/f78 0 2024-03-11T19:56:36.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/906: mkdir d6/d28/d12d 0 2024-03-11T19:56:36.814 INFO:tasks.workunit.client.0.smithi107.stdout:1/907: rmdir d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/dd6 0 2024-03-11T19:56:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:1/908: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107/f12e x:0 0 0 2024-03-11T19:56:36.826 INFO:tasks.workunit.client.0.smithi107.stdout:2/910: dwrite d1/d21/f96 [0,4194304] 0 2024-03-11T19:56:36.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/911: creat d1/d1a/d1e/f130 x:0 0 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/879: dwrite dd/f2e [0,4194304] 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/880: truncate dd/d16/f33 1969491 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/881: fdatasync dd/f2e 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/882: stat dd/d13/d18/d81/ddc/fd1 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/883: read dd/d16/f19 [5124846,103315] 0 2024-03-11T19:56:36.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/884: read - dd/de/d24/d28/d47/dbb/fd5 zero size 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/885: getdents dd/de/dfc/d10f/d46/de7 0 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/886: write dd/d13/d32/fe0 [725803,95929] 0 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/887: creat dd/dac/f112 x:0 0 0 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/888: creat dd/de/d24/f113 x:0 0 0 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/889: chown dd/de/d1b/d20/c54 230631147 1 2024-03-11T19:56:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/890: write dd/de/dfc/d10f/f60 [62490,103688] 0 2024-03-11T19:56:36.874 INFO:tasks.workunit.client.0.smithi107.stdout:2/912: dwrite d1/d1a/f2c [0,4194304] 0 2024-03-11T19:56:36.879 INFO:tasks.workunit.client.0.smithi107.stdout:2/913: dread d1/d21/fa7 [0,4194304] 0 2024-03-11T19:56:36.881 INFO:tasks.workunit.client.0.smithi107.stdout:2/914: creat d1/d21/d35/d67/f131 x:0 0 0 2024-03-11T19:56:36.884 INFO:tasks.workunit.client.0.smithi107.stdout:8/891: dwrite dd/d13/d18/fc1 [0,4194304] 0 2024-03-11T19:56:36.884 INFO:tasks.workunit.client.0.smithi107.stdout:8/892: chown dd/d13/d32/f89 3 1 2024-03-11T19:56:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/894: sync 2024-03-11T19:56:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:4/883: sync 2024-03-11T19:56:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/910: sync 2024-03-11T19:56:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/911: chown d10/d16/d1f/d51/d76/f9f 254948 1 2024-03-11T19:56:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:4/884: dread - d4/d7/db/fc2 zero size 2024-03-11T19:56:36.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/885: symlink d4/d7/d2f/d3b/d101/d93/dc3/l115 0 2024-03-11T19:56:36.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/886: symlink d4/d7/d2f/d3b/d101/d24/d26/d29/d36/l116 0 2024-03-11T19:56:36.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/887: readlink d4/d7/d2f/d3b/d101/d19/d75/d8e/l9e 0 2024-03-11T19:56:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/888: creat d4/d7/da0/f117 x:0 0 0 2024-03-11T19:56:36.940 INFO:tasks.workunit.client.0.smithi107.stdout:9/895: dread d4/d9/dae/ffa [0,4194304] 0 2024-03-11T19:56:36.940 INFO:tasks.workunit.client.0.smithi107.stdout:9/896: chown d4/ld 97068924 1 2024-03-11T19:56:36.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/897: getdents d4/d1d/d1f/d28 0 2024-03-11T19:56:36.944 INFO:tasks.workunit.client.0.smithi107.stdout:9/898: creat d4/d1c/d37/f135 x:0 0 0 2024-03-11T19:56:36.986 INFO:tasks.workunit.client.0.smithi107.stdout:4/889: dwrite d4/d7/d2f/d3c/f73 [0,4194304] 0 2024-03-11T19:56:36.988 INFO:tasks.workunit.client.0.smithi107.stdout:4/890: creat d4/d7/d2f/d3b/d101/d93/f118 x:0 0 0 2024-03-11T19:56:36.988 INFO:tasks.workunit.client.0.smithi107.stdout:9/899: write d4/d1d/d1f/d28/d2c/f5b [2975867,872] 0 2024-03-11T19:56:36.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/912: dwrite d10/d16/d24/d2e/d9d/f92 [0,4194304] 0 2024-03-11T19:56:36.991 INFO:tasks.workunit.client.0.smithi107.stdout:4/891: rename d4/d7/d2f/d3b/d101/d19/d9a to d4/d7/d2f/d3b/d101/d24/d26/d119 0 2024-03-11T19:56:36.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/913: creat d10/d16/d2b/f122 x:0 0 0 2024-03-11T19:56:36.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/914: truncate d10/d3d/d54/fd2 825376 0 2024-03-11T19:56:36.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/915: link d10/d16/d24/l3f d10/d16/d2b/d39/d6a/l123 0 2024-03-11T19:56:36.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/916: getdents d10/d16/d1f/d51/d9c/db6/d102 0 2024-03-11T19:56:36.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/917: symlink d10/d16/d2b/d8d/l124 0 2024-03-11T19:56:37.009 INFO:tasks.workunit.client.0.smithi107.stdout:1/909: sync 2024-03-11T19:56:37.009 INFO:tasks.workunit.client.0.smithi107.stdout:8/893: sync 2024-03-11T19:56:37.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/915: sync 2024-03-11T19:56:37.010 INFO:tasks.workunit.client.0.smithi107.stdout:2/916: write d1/d21/f28 [2123424,113700] 0 2024-03-11T19:56:37.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:36 smithi122 ceph-mon[36397]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 781 KiB/s wr, 43 op/s 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:6/918: unlink d10/d16/d2b/d39/l81 0 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/910: mknod d6/d7e/d81/d27/d2c/d72/dc3/dd4/d82/c12f 0 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/911: dread - d6/d7e/d81/d27/d2c/d32/dc1/dc9/f11a zero size 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/912: dread - d6/d7e/d81/d27/d2c/d72/dc3/dd4/ffd zero size 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/913: readlink d6/d7e/d81/d27/d2c/d32/dc1/dc9/d11c/l129 0 2024-03-11T19:56:37.011 INFO:tasks.workunit.client.0.smithi107.stdout:2/917: read d1/d1a/d34/d4f/d63/db6/f108 [176352,22191] 0 2024-03-11T19:56:37.015 INFO:tasks.workunit.client.0.smithi107.stdout:2/918: link d1/d38/d75/dd7/l11e d1/d1a/d1e/l132 0 2024-03-11T19:56:37.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/914: creat d6/d7e/d81/d27/d2c/d32/dc1/df7/f130 x:0 0 0 2024-03-11T19:56:37.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/915: mknod d6/d28/dc4/c131 0 2024-03-11T19:56:37.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:36 smithi145 ceph-mon[36730]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 781 KiB/s wr, 43 op/s 2024-03-11T19:56:37.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:36 smithi107 ceph-mon[31712]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 781 KiB/s wr, 43 op/s 2024-03-11T19:56:37.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/892: dwrite f3 [4194304,4194304] 0 2024-03-11T19:56:37.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/893: dread - d4/d7/d2f/d3b/d101/f109 zero size 2024-03-11T19:56:37.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/919: dread d1/d1d/d64/f99 [0,4194304] 0 2024-03-11T19:56:37.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/920: chown d1/d38/d75/d76/f10c 4554176 1 2024-03-11T19:56:37.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/921: creat d1/d1a/d34/f133 x:0 0 0 2024-03-11T19:56:37.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/922: write d1/d1a/d34/f133 [439685,113654] 0 2024-03-11T19:56:37.056 INFO:tasks.workunit.client.0.smithi107.stdout:2/923: mkdir d1/d1a/d8e/d134 0 2024-03-11T19:56:37.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/924: symlink d1/d38/l135 0 2024-03-11T19:56:37.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/925: symlink d1/d1d/d80/l136 0 2024-03-11T19:56:37.063 INFO:tasks.workunit.client.0.smithi107.stdout:2/926: rename d1/d1a/d1e/d55/f87 to d1/d21/d35/dfc/f137 0 2024-03-11T19:56:37.064 INFO:tasks.workunit.client.0.smithi107.stdout:2/927: unlink d1/d38/d75/fa0 0 2024-03-11T19:56:37.066 INFO:tasks.workunit.client.0.smithi107.stdout:2/928: getdents d1/d1d/d64/d97/dd1 0 2024-03-11T19:56:37.080 INFO:tasks.workunit.client.0.smithi107.stdout:6/919: dwrite d10/d3d/d54/fc9 [0,4194304] 0 2024-03-11T19:56:37.081 INFO:tasks.workunit.client.0.smithi107.stdout:6/920: symlink d10/d16/d1f/d51/d9c/l125 0 2024-03-11T19:56:37.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/929: write d1/d1d/d64/f99 [2334133,10848] 0 2024-03-11T19:56:37.097 INFO:tasks.workunit.client.0.smithi107.stdout:4/894: dwrite d4/d7/da0/fa1 [4194304,4194304] 0 2024-03-11T19:56:37.100 INFO:tasks.workunit.client.0.smithi107.stdout:4/895: truncate d4/d7/d2f/d3b/d101/d19/d75/fb5 5223763 0 2024-03-11T19:56:37.112 INFO:tasks.workunit.client.0.smithi107.stdout:4/896: dread d4/d7/d2f/d3b/f7e [0,4194304] 0 2024-03-11T19:56:37.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/897: creat d4/d7/d2f/d3b/d101/d24/f11a x:0 0 0 2024-03-11T19:56:37.116 INFO:tasks.workunit.client.0.smithi107.stdout:9/900: dwrite d4/d9/dae/f77 [0,4194304] 0 2024-03-11T19:56:37.117 INFO:tasks.workunit.client.0.smithi107.stdout:8/894: dwrite dd/d13/d18/d81/ddc/d90/d96/ff6 [0,4194304] 0 2024-03-11T19:56:37.118 INFO:tasks.workunit.client.0.smithi107.stdout:9/901: symlink d4/d1c/l136 0 2024-03-11T19:56:37.119 INFO:tasks.workunit.client.0.smithi107.stdout:9/902: fsync d4/d1c/f9e 0 2024-03-11T19:56:37.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/903: creat d4/d1d/f137 x:0 0 0 2024-03-11T19:56:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:8/895: mknod dd/de/c114 0 2024-03-11T19:56:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:9/904: fdatasync d4/d1d/d5c/d97/fb9 0 2024-03-11T19:56:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:9/905: creat d4/d1d/d4b/dfd/f138 x:0 0 0 2024-03-11T19:56:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:9/906: readlink d4/d9/dae/d6d/l130 0 2024-03-11T19:56:37.133 INFO:tasks.workunit.client.0.smithi107.stdout:8/896: mkdir dd/de/d24/d115 0 2024-03-11T19:56:37.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/916: dwrite d6/d7e/d81/d27/d2c/f6a [0,4194304] 0 2024-03-11T19:56:37.149 INFO:tasks.workunit.client.0.smithi107.stdout:1/917: mknod d6/d7e/da5/c132 0 2024-03-11T19:56:37.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/918: symlink d6/d7e/d81/d27/d2c/d7d/dd8/l133 0 2024-03-11T19:56:37.152 INFO:tasks.workunit.client.0.smithi107.stdout:1/919: mknod d6/d7e/d81/d27/d2c/d32/dc1/dc9/d11c/c134 0 2024-03-11T19:56:37.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/920: symlink d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/l135 0 2024-03-11T19:56:37.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/921: truncate d6/d7e/d81/f29 3647915 0 2024-03-11T19:56:37.156 INFO:tasks.workunit.client.0.smithi107.stdout:9/907: dread d4/d1d/d5c/d64/f84 [0,4194304] 0 2024-03-11T19:56:37.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/922: link d6/d7e/da5/c132 d6/d7e/d81/d27/d2c/d32/dc1/dc9/dfe/c136 0 2024-03-11T19:56:37.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/923: chown d6/d7e/d81/d27/d2c/d7d/dd8/cec 1570037853 1 2024-03-11T19:56:37.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/908: getdents d4 0 2024-03-11T19:56:37.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/924: creat d6/f137 x:0 0 0 2024-03-11T19:56:37.161 INFO:tasks.workunit.client.0.smithi107.stdout:6/921: dwrite d10/d16/d2b/f59 [0,4194304] 0 2024-03-11T19:56:37.161 INFO:tasks.workunit.client.0.smithi107.stdout:6/922: truncate d10/d3d/d54/f114 334148 0 2024-03-11T19:56:37.161 INFO:tasks.workunit.client.0.smithi107.stdout:9/909: rmdir d4/d1d/d1f/d28/d5d 39 2024-03-11T19:56:37.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/910: truncate d4/d9/d4d/fd6 2092012 0 2024-03-11T19:56:37.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/930: dwrite d1/d1a/d1e/d26/f48 [0,4194304] 0 2024-03-11T19:56:37.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/931: chown d1/d21/d35/d67/l10e 12 1 2024-03-11T19:56:37.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/932: write d1/d21/d35/f8d [4926612,114758] 0 2024-03-11T19:56:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:1/925: dread d6/d7e/d81/d27/d2c/d32/d58/fd0 [0,4194304] 0 2024-03-11T19:56:37.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/933: unlink d1/d1a/d34/d4f/d63/db6/f108 0 2024-03-11T19:56:37.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/911: rmdir d4/d1d/d1f/d28/d2c/d96 39 2024-03-11T19:56:37.186 INFO:tasks.workunit.client.0.smithi107.stdout:4/898: dwrite d4/d7/d2f/d3b/d101/d24/d26/d29/d36/f57 [4194304,4194304] 0 2024-03-11T19:56:37.187 INFO:tasks.workunit.client.0.smithi107.stdout:9/912: creat d4/d9/d4d/f139 x:0 0 0 2024-03-11T19:56:37.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/934: getdents d1/d1a/d34/d4f/d63/db6 0 2024-03-11T19:56:37.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/935: truncate d1/d1a/d34/d4f/d63/db6/fbd 671323 0 2024-03-11T19:56:37.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/936: fdatasync d1/f15 0 2024-03-11T19:56:37.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/937: truncate d1/d1a/fa3 873603 0 2024-03-11T19:56:37.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/899: mkdir d4/d7/d2f/d3b/d101/d24/d11b 0 2024-03-11T19:56:37.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/900: truncate d4/d7/d2f/d3b/d101/d19/d75/fe6 331838 0 2024-03-11T19:56:37.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/901: chown d4/d7/d2f/d3b/d101/d24/d26/d29/l46 128132789 1 2024-03-11T19:56:37.199 INFO:tasks.workunit.client.0.smithi107.stdout:9/913: creat d4/d106/f13a x:0 0 0 2024-03-11T19:56:37.199 INFO:tasks.workunit.client.0.smithi107.stdout:9/914: chown d4/d1d/d5c/fa6 18506538 1 2024-03-11T19:56:37.199 INFO:tasks.workunit.client.0.smithi107.stdout:9/915: fdatasync d4/d1d/d5c/fde 0 2024-03-11T19:56:37.199 INFO:tasks.workunit.client.0.smithi107.stdout:9/916: fdatasync d4/fcb 0 2024-03-11T19:56:37.199 INFO:tasks.workunit.client.0.smithi107.stdout:9/917: chown d4/d1d/d1f/f39 0 1 2024-03-11T19:56:37.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/938: creat d1/d1a/d1e/d26/f138 x:0 0 0 2024-03-11T19:56:37.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/926: link d6/d7e/d81/d27/d2c/d32/l6e d6/d7e/d81/d27/d2c/d32/d58/l138 0 2024-03-11T19:56:37.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/927: chown d6/f1d 12118 1 2024-03-11T19:56:37.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/928: symlink d6/d7e/d81/d27/d2c/l139 0 2024-03-11T19:56:37.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/902: creat d4/d7/d2f/d3b/d101/d24/d26/d29/f11c x:0 0 0 2024-03-11T19:56:37.206 INFO:tasks.workunit.client.0.smithi107.stdout:1/929: mkdir d6/d7e/d81/d27/dfa/dff/d13a 0 2024-03-11T19:56:37.206 INFO:tasks.workunit.client.0.smithi107.stdout:1/930: dread - d6/d7e/d81/d27/d2c/d72/dc3/dd4/da2/d10c/d128/f12c zero size 2024-03-11T19:56:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:1/931: link d6/d7e/d81/d27/d2c/d32/f8d d6/d7e/d81/d27/d2c/d32/f13b 0 2024-03-11T19:56:37.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/939: rename d1/d21/d35/f54 to d1/d1a/d1e/d103/f139 0 2024-03-11T19:56:37.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/932: rename d6/d28/laa to d6/d7e/d81/d27/d2c/d32/dc1/dc9/dfe/l13c 0 2024-03-11T19:56:37.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/933: fdatasync d6/d28/f10a 0 2024-03-11T19:56:37.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/934: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/def/f13d x:0 0 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/935: stat d6/d7e/da4 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/936: write d6/d7e/d81/d27/d2c/d7d/dd8/ff1 [4576496,75425] 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:2/940: mknod d1/d1d/d64/d97/dd1/c13a 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/937: truncate d6/d7e/d81/f21 957415 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/938: write d6/d7e/da5/dd9/ffc [1049094,50669] 0 2024-03-11T19:56:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/939: readlink d6/d7e/d81/d27/d2c/d72/dc3/dd4/l57 0 2024-03-11T19:56:37.217 INFO:tasks.workunit.client.0.smithi107.stdout:2/941: truncate d1/d1a/d34/d4f/d63/f66 4053599 0 2024-03-11T19:56:37.218 INFO:tasks.workunit.client.0.smithi107.stdout:1/940: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107/f13e x:0 0 0 2024-03-11T19:56:37.219 INFO:tasks.workunit.client.0.smithi107.stdout:6/923: dwrite d10/d16/d24/d2e/d9d/fa8 [4194304,4194304] 0 2024-03-11T19:56:37.220 INFO:tasks.workunit.client.0.smithi107.stdout:1/941: rename d6/d7e/d81/d27/d2c/d32/dc1/dc9/d11c/c134 to d6/d86/c13f 0 2024-03-11T19:56:37.220 INFO:tasks.workunit.client.0.smithi107.stdout:1/942: truncate d6/d7e/d81/d27/f47 8451424 0 2024-03-11T19:56:37.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/918: truncate d4/d1d/d5c/d64/db3/d100/f129 4204315 0 2024-03-11T19:56:37.223 INFO:tasks.workunit.client.0.smithi107.stdout:2/942: symlink d1/d38/d75/d76/l13b 0 2024-03-11T19:56:37.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/924: symlink d10/d16/d1f/d51/l126 0 2024-03-11T19:56:37.226 INFO:tasks.workunit.client.0.smithi107.stdout:6/925: mkdir d10/d19/d11c/d127 0 2024-03-11T19:56:37.226 INFO:tasks.workunit.client.0.smithi107.stdout:1/943: creat d6/d28/dc4/f140 x:0 0 0 2024-03-11T19:56:37.227 INFO:tasks.workunit.client.0.smithi107.stdout:2/943: stat d1/d1a/d34/lb9 0 2024-03-11T19:56:37.228 INFO:tasks.workunit.client.0.smithi107.stdout:1/944: mkdir d6/d7e/da4/d141 0 2024-03-11T19:56:37.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/945: creat d6/d5b/f142 x:0 0 0 2024-03-11T19:56:37.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/926: creat d10/d16/d1f/d51/d76/d7f/dbd/f128 x:0 0 0 2024-03-11T19:56:37.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/946: dread - d6/d7e/d81/d27/d2c/d32/dc1/dc9/def/f13d zero size 2024-03-11T19:56:37.231 INFO:tasks.workunit.client.0.smithi107.stdout:6/927: creat d10/f129 x:0 0 0 2024-03-11T19:56:37.232 INFO:tasks.workunit.client.0.smithi107.stdout:2/944: mknod d1/d1a/d1e/d103/c13c 0 2024-03-11T19:56:37.233 INFO:tasks.workunit.client.0.smithi107.stdout:2/945: creat d1/f13d x:0 0 0 2024-03-11T19:56:37.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/947: dwrite d6/d7e/da5/dd9/ffc [0,4194304] 0 2024-03-11T19:56:37.267 INFO:tasks.workunit.client.0.smithi107.stdout:1/948: creat d6/d7e/d81/d27/dfa/dbb/d11b/f143 x:0 0 0 2024-03-11T19:56:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/928: dwrite d10/d16/d1f/d51/d9c/fb0 [0,4194304] 0 2024-03-11T19:56:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/929: write d10/d3d/d54/f80 [1037501,61888] 0 2024-03-11T19:56:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/930: chown d10/d16/d1f/d51/d76/fdf 3106113 1 2024-03-11T19:56:37.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/946: dwrite d1/d1d/d64/f99 [0,4194304] 0 2024-03-11T19:56:37.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/931: dread d10/d19/f77 [0,4194304] 0 2024-03-11T19:56:37.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/932: symlink d10/d16/d1f/d51/d7a/l12a 0 2024-03-11T19:56:37.320 INFO:tasks.workunit.client.0.smithi107.stdout:2/947: dwrite d1/d1a/d8e/fec [0,4194304] 0 2024-03-11T19:56:37.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/933: dread d10/d16/fcd [0,4194304] 0 2024-03-11T19:56:37.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/934: getdents d10/d16/d1f/d51/d76/d7f/dbd 0 2024-03-11T19:56:37.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/935: mknod d10/d16/d2b/da5/c12b 0 2024-03-11T19:56:37.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/936: stat d10/d19/f89 0 2024-03-11T19:56:37.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/937: chown d10/d16/d24/f68 29 1 2024-03-11T19:56:37.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/897: sync 2024-03-11T19:56:37.361 INFO:tasks.workunit.client.0.smithi107.stdout:6/938: write d10/d16/d1f/d51/d9c/f104 [2527229,130915] 0 2024-03-11T19:56:37.361 INFO:tasks.workunit.client.0.smithi107.stdout:6/939: creat d10/d19/d3b/ddb/f12c x:0 0 0 2024-03-11T19:56:37.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/940: dread d10/d3d/d54/fd2 [0,4194304] 0 2024-03-11T19:56:37.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/941: chown d10/d19/d3b/d70/fd8 92771073 1 2024-03-11T19:56:37.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/942: write d10/d16/d2b/d39/d6a/f120 [658808,1373] 0 2024-03-11T19:56:37.368 INFO:tasks.workunit.client.0.smithi107.stdout:2/948: dwrite d1/d1a/d8e/f120 [0,4194304] 0 2024-03-11T19:56:37.370 INFO:tasks.workunit.client.0.smithi107.stdout:2/949: mkdir d1/d1a/d1e/d26/dd6/df4/d12b/d13e 0 2024-03-11T19:56:37.398 INFO:tasks.workunit.client.0.smithi107.stdout:6/943: rename d10/d16/d1f/d51/d76/f93 to d10/d16/d2b/d39/f12d 0 2024-03-11T19:56:37.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/944: mknod d10/d19/d3b/d70/d8a/c12e 0 2024-03-11T19:56:37.403 INFO:tasks.workunit.client.0.smithi107.stdout:6/945: truncate d10/d16/d2b/f59 2793279 0 2024-03-11T19:56:37.405 INFO:tasks.workunit.client.0.smithi107.stdout:6/946: creat d10/dee/f12f x:0 0 0 2024-03-11T19:56:37.405 INFO:tasks.workunit.client.0.smithi107.stdout:6/947: dread - d10/d16/d2b/da5/d118/f121 zero size 2024-03-11T19:56:37.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/948: creat d10/d16/d24/d2e/ded/f130 x:0 0 0 2024-03-11T19:56:37.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/949: fdatasync d10/d19/f77 0 2024-03-11T19:56:37.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/950: creat d10/d16/d2b/d8d/f131 x:0 0 0 2024-03-11T19:56:37.409 INFO:tasks.workunit.client.0.smithi107.stdout:6/951: unlink d10/d16/d1f/d51/d7a/dba/fc7 0 2024-03-11T19:56:37.409 INFO:tasks.workunit.client.0.smithi107.stdout:6/952: getdents d10/d19/d11c/d127 0 2024-03-11T19:56:37.409 INFO:tasks.workunit.client.0.smithi107.stdout:6/953: stat d10/d16/fab 0 2024-03-11T19:56:37.413 INFO:tasks.workunit.client.0.smithi107.stdout:6/954: symlink d10/d16/d2b/da5/d118/l132 0 2024-03-11T19:56:37.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/955: write d10/d3d/d54/f117 [945377,76787] 0 2024-03-11T19:56:37.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/956: write d10/d16/d2b/d39/f12d [4827661,55481] 0 2024-03-11T19:56:37.418 INFO:tasks.workunit.client.0.smithi107.stdout:8/898: dwrite dd/d13/dd7/ffb [0,4194304] 0 2024-03-11T19:56:37.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/899: truncate dd/d13/d32/d51/fb3 4335365 0 2024-03-11T19:56:37.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/900: creat dd/d13/f116 x:0 0 0 2024-03-11T19:56:37.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/950: dwrite d1/f2b [0,4194304] 0 2024-03-11T19:56:37.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/951: readlink d1/d38/l135 0 2024-03-11T19:56:37.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/952: mknod d1/d1a/d8e/c13f 0 2024-03-11T19:56:37.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/953: dread - d1/d38/d75/d76/f118 zero size 2024-03-11T19:56:37.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/954: fdatasync d1/f13d 0 2024-03-11T19:56:37.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/955: mknod d1/d21/c140 0 2024-03-11T19:56:37.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/957: dread d10/d19/fa6 [0,4194304] 0 2024-03-11T19:56:37.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/958: chown d10/d19/f74 1 1 2024-03-11T19:56:37.440 INFO:tasks.workunit.client.0.smithi107.stdout:6/959: creat d10/d16/d1f/d51/d7a/f133 x:0 0 0 2024-03-11T19:56:37.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/960: mknod d10/d16/d1f/c134 0 2024-03-11T19:56:37.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/961: truncate d10/d16/d1f/f25 1516658 0 2024-03-11T19:56:37.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/962: fdatasync d10/d16/d2b/da5/d118/f121 0 2024-03-11T19:56:37.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/901: rename dd/d13/dd7 to dd/d13/d18/d81/ddc/de8/d117 0 2024-03-11T19:56:37.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/902: fsync dd/d13/d32/fe0 0 2024-03-11T19:56:37.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/903: fdatasync dd/de/d1b/d20/d56/fb8 0 2024-03-11T19:56:37.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/904: chown dd/dac/f110 816217 1 2024-03-11T19:56:37.447 INFO:tasks.workunit.client.0.smithi107.stdout:8/905: mkdir dd/d13/d18/dd0/d118 0 2024-03-11T19:56:37.452 INFO:tasks.workunit.client.0.smithi107.stdout:1/949: sync 2024-03-11T19:56:37.452 INFO:tasks.workunit.client.0.smithi107.stdout:1/950: write d6/d7e/d81/d27/f38 [5585734,30750] 0 2024-03-11T19:56:37.452 INFO:tasks.workunit.client.0.smithi107.stdout:8/906: mkdir dd/de/d24/d115/d119 0 2024-03-11T19:56:37.452 INFO:tasks.workunit.client.0.smithi107.stdout:8/907: rmdir dd/de/d1b/d20/d56/da4/dee 39 2024-03-11T19:56:37.452 INFO:tasks.workunit.client.0.smithi107.stdout:9/919: sync 2024-03-11T19:56:37.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/903: sync 2024-03-11T19:56:37.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/920: write d4/d1c/d47/f9c [780065,122788] 0 2024-03-11T19:56:37.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/908: creat dd/de/d1b/d20/d5e/f11a x:0 0 0 2024-03-11T19:56:37.453 INFO:tasks.workunit.client.0.smithi107.stdout:2/956: dread d1/d38/d75/f84 [0,4194304] 0 2024-03-11T19:56:37.457 INFO:tasks.workunit.client.0.smithi107.stdout:9/921: creat d4/d1c/d37/dd9/f13b x:0 0 0 2024-03-11T19:56:37.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/957: symlink d1/d1a/d34/d4f/d63/d100/l141 0 2024-03-11T19:56:37.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/951: dread d6/d28/f10a [0,4194304] 0 2024-03-11T19:56:37.468 INFO:tasks.workunit.client.0.smithi107.stdout:2/958: dread d1/d21/d35/f5c [0,4194304] 0 2024-03-11T19:56:37.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/959: read d1/d1a/d34/d4f/d63/d100/f107 [225284,85419] 0 2024-03-11T19:56:37.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/960: creat d1/d1a/d1e/d26/dd6/dfa/f142 x:0 0 0 2024-03-11T19:56:37.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/961: symlink d1/d1a/d1e/d26/dd6/l143 0 2024-03-11T19:56:37.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/952: dwrite d6/d28/fb2 [4194304,4194304] 0 2024-03-11T19:56:37.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/953: truncate d6/f113 298455 0 2024-03-11T19:56:37.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/954: read - d6/d7e/da5/dd9/f127 zero size 2024-03-11T19:56:37.516 INFO:tasks.workunit.client.0.smithi107.stdout:1/955: rename d6/d28/dc4/c8e to d6/d7e/d81/d27/d2c/d32/d58/c144 0 2024-03-11T19:56:37.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/956: symlink d6/d7e/d81/d27/d2c/d72/l145 0 2024-03-11T19:56:37.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/957: rename d6/d7e/d81/d27/d2c/d7d/ca9 to d6/d7e/d81/d27/dfa/dbb/c146 0 2024-03-11T19:56:37.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/963: dwrite d10/d16/d24/d2e/ded/f130 [0,4194304] 0 2024-03-11T19:56:37.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/958: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/f147 x:0 0 0 2024-03-11T19:56:37.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/964: creat d10/d19/d38/d109/f135 x:0 0 0 2024-03-11T19:56:37.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/922: dwrite d4/d1d/d5c/d97/fe0 [0,4194304] 0 2024-03-11T19:56:37.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/923: chown d4/d1c/d47/d8a/l99 338 1 2024-03-11T19:56:37.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/924: fsync d4/d1d/d1f/d28/fe7 0 2024-03-11T19:56:37.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/925: chown d4/d1c/d37 23 1 2024-03-11T19:56:37.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/926: fsync d4/d1d/d4b/dfd/f12c 0 2024-03-11T19:56:37.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/959: rmdir d6/d7e/da4/d141 0 2024-03-11T19:56:37.527 INFO:tasks.workunit.client.0.smithi107.stdout:1/960: rmdir d6/d7e/d81/d27/d2c/d32/dcd 39 2024-03-11T19:56:37.530 INFO:tasks.workunit.client.0.smithi107.stdout:9/927: unlink d4/d1d/d1f/d28/d2c/c7b 0 2024-03-11T19:56:37.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/904: dwrite d4/fbe [0,4194304] 0 2024-03-11T19:56:37.532 INFO:tasks.workunit.client.0.smithi107.stdout:9/928: creat d4/d1d/d5c/f13c x:0 0 0 2024-03-11T19:56:37.533 INFO:tasks.workunit.client.0.smithi107.stdout:1/961: rename d6/l18 to d6/l148 0 2024-03-11T19:56:37.533 INFO:tasks.workunit.client.0.smithi107.stdout:1/962: write d6/d7e/d81/d27/d2c/d32/dc1/dc9/dfe/f43 [2739245,111129] 0 2024-03-11T19:56:37.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/905: mkdir d4/d7/da0/d11d 0 2024-03-11T19:56:37.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/929: unlink d4/d1c/d37/f75 0 2024-03-11T19:56:37.536 INFO:tasks.workunit.client.0.smithi107.stdout:1/963: creat d6/d118/f149 x:0 0 0 2024-03-11T19:56:37.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/930: symlink d4/d1d/d5c/d97/ddc/l13d 0 2024-03-11T19:56:37.538 INFO:tasks.workunit.client.0.smithi107.stdout:1/964: link d6/d7e/d81/d27/d2c/d7d/dd8/ff1 d6/d28/d12d/f14a 0 2024-03-11T19:56:37.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/931: creat d4/d1d/d5c/f13e x:0 0 0 2024-03-11T19:56:37.540 INFO:tasks.workunit.client.0.smithi107.stdout:2/962: dwrite d1/d1d/d64/f99 [0,4194304] 0 2024-03-11T19:56:37.540 INFO:tasks.workunit.client.0.smithi107.stdout:1/965: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107/f14b x:0 0 0 2024-03-11T19:56:37.541 INFO:tasks.workunit.client.0.smithi107.stdout:1/966: readlink d6/d5b/l6d 0 2024-03-11T19:56:37.543 INFO:tasks.workunit.client.0.smithi107.stdout:1/967: mkdir d6/d118/d14c 0 2024-03-11T19:56:37.543 INFO:tasks.workunit.client.0.smithi107.stdout:9/932: mkdir d4/d1d/d5c/d13f 0 2024-03-11T19:56:37.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/963: creat d1/d1d/d64/f144 x:0 0 0 2024-03-11T19:56:37.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/964: creat d1/d38/d75/d76/f145 x:0 0 0 2024-03-11T19:56:37.547 INFO:tasks.workunit.client.0.smithi107.stdout:2/965: link d1/f10 d1/d1a/d34/d4f/d63/db6/f146 0 2024-03-11T19:56:37.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/965: rename d10/d16/d24/cbc to d10/d16/d1f/d51/d7a/c136 0 2024-03-11T19:56:37.575 INFO:tasks.workunit.client.0.smithi107.stdout:6/966: creat d10/d16/d24/d2e/ded/f137 x:0 0 0 2024-03-11T19:56:37.575 INFO:tasks.workunit.client.0.smithi107.stdout:6/967: chown d10/d3d/d54/fd2 589325 1 2024-03-11T19:56:37.580 INFO:tasks.workunit.client.0.smithi107.stdout:4/906: dwrite d4/d7/da0/fb3 [0,4194304] 0 2024-03-11T19:56:37.580 INFO:tasks.workunit.client.0.smithi107.stdout:4/907: readlink d4/d7/d2f/d3b/d101/d24/d26/d29/d36/l90 0 2024-03-11T19:56:37.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/968: dwrite d6/d7e/d81/d27/d2c/d72/f75 [0,4194304] 0 2024-03-11T19:56:37.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/969: chown d6/d7e/d81/d27/d2c/d72/dc3/dd4/dd3/f147 193249111 1 2024-03-11T19:56:37.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/970: dread - d6/d118/f149 zero size 2024-03-11T19:56:37.602 INFO:tasks.workunit.client.0.smithi107.stdout:2/966: dwrite d1/d38/d75/fb0 [4194304,4194304] 0 2024-03-11T19:56:37.602 INFO:tasks.workunit.client.0.smithi107.stdout:2/967: creat d1/d1a/d34/f147 x:0 0 0 2024-03-11T19:56:37.602 INFO:tasks.workunit.client.0.smithi107.stdout:2/968: creat d1/d1d/d64/f148 x:0 0 0 2024-03-11T19:56:37.605 INFO:tasks.workunit.client.0.smithi107.stdout:2/969: mknod d1/d1a/d34/c149 0 2024-03-11T19:56:37.609 INFO:tasks.workunit.client.0.smithi107.stdout:2/970: mkdir d1/d1a/d34/d4f/d63/d14a 0 2024-03-11T19:56:37.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/933: rename d4/d106/f13a to d4/d9/d9f/dd8/f140 0 2024-03-11T19:56:37.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/934: truncate d4/d9/dae/d6d/ff7 943861 0 2024-03-11T19:56:37.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/935: readlink d4/d1d/d1f/d28/d5d/l6b 0 2024-03-11T19:56:37.616 INFO:tasks.workunit.client.0.smithi107.stdout:2/971: dread d1/d21/d35/dfc/f137 [0,4194304] 0 2024-03-11T19:56:37.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/972: creat d1/d1a/d34/d4f/d63/db6/f14b x:0 0 0 2024-03-11T19:56:37.626 INFO:tasks.workunit.client.0.smithi107.stdout:1/971: write d6/d7e/d81/d27/f47 [104271,118532] 0 2024-03-11T19:56:37.628 INFO:tasks.workunit.client.0.smithi107.stdout:1/972: link d6/d7e/d81/d27/d2c/l2d d6/d118/d14c/l14d 0 2024-03-11T19:56:37.628 INFO:tasks.workunit.client.0.smithi107.stdout:4/908: dwrite d4/d7/f8 [4194304,4194304] 0 2024-03-11T19:56:37.628 INFO:tasks.workunit.client.0.smithi107.stdout:1/973: creat d6/d28/d12d/f14e x:0 0 0 2024-03-11T19:56:37.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/909: creat d4/d7/d2f/d33/f11e x:0 0 0 2024-03-11T19:56:37.642 INFO:tasks.workunit.client.0.smithi107.stdout:6/968: rename d10/d16/d1f/d51/d76/d7f/dbd to d10/d19/d3b/ddb/d138 0 2024-03-11T19:56:37.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/909: sync 2024-03-11T19:56:37.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/936: rename d4/d1c/f7e to d4/d1d/d4b/f141 0 2024-03-11T19:56:37.647 INFO:tasks.workunit.client.0.smithi107.stdout:9/937: rename d4/d1d/d5c/f13e to d4/d1d/d5c/f142 0 2024-03-11T19:56:37.649 INFO:tasks.workunit.client.0.smithi107.stdout:9/938: symlink d4/d1c/d37/dd9/de9/df8/l143 0 2024-03-11T19:56:37.652 INFO:tasks.workunit.client.0.smithi107.stdout:9/939: rename d4/d1d/d1f/d78/d118/f132 to d4/f144 0 2024-03-11T19:56:37.666 INFO:tasks.workunit.client.0.smithi107.stdout:2/973: dwrite d1/d1a/d34/f10d [0,4194304] 0 2024-03-11T19:56:37.668 INFO:tasks.workunit.client.0.smithi107.stdout:2/974: creat d1/d1a/d1e/d26/dd6/df4/f14c x:0 0 0 2024-03-11T19:56:37.669 INFO:tasks.workunit.client.0.smithi107.stdout:4/910: write d4/d7/da0/fb3 [2663586,62492] 0 2024-03-11T19:56:37.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/910: dread dd/d13/d18/f1a [4194304,4194304] 0 2024-03-11T19:56:37.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/975: dread d1/d1a/d1e/d41/f51 [0,4194304] 0 2024-03-11T19:56:37.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/976: fdatasync d1/d1d/d64/f144 0 2024-03-11T19:56:37.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/969: dwrite d10/d3d/d54/f11b [0,4194304] 0 2024-03-11T19:56:37.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/970: chown d10/d16/d2b/d39/d6a/d6e/def/db8/fcc 3316269 1 2024-03-11T19:56:37.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/971: write d10/dee/f12f [441052,106397] 0 2024-03-11T19:56:37.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/977: rename d1/d21/d35/dfc/d125 to d1/d1d/d64/d14d 0 2024-03-11T19:56:37.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/978: readlink d1/d38/d75/dd7/l10b 0 2024-03-11T19:56:37.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/979: write d1/d1a/d1e/f130 [707153,37481] 0 2024-03-11T19:56:37.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/940: read d4/d1d/d5c/d97/fe0 [3526888,65542] 0 2024-03-11T19:56:37.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/941: fdatasync d4/d1d/d1f/d78/f74 0 2024-03-11T19:56:37.696 INFO:tasks.workunit.client.0.smithi107.stdout:6/972: getdents d10/d16/d2b/d39 0 2024-03-11T19:56:37.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/973: rename d10/d19/d3b/d70/le3 to d10/d19/d38/d109/l139 0 2024-03-11T19:56:37.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/974: dwrite d6/d7e/d81/d27/d2c/d72/dc3/dd4/ffd [0,4194304] 0 2024-03-11T19:56:37.742 INFO:tasks.workunit.client.0.smithi107.stdout:9/942: dread d4/d1d/d1f/d28/d2c/f5b [0,4194304] 0 2024-03-11T19:56:37.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/911: dwrite d4/d7/d2f/ff6 [0,4194304] 0 2024-03-11T19:56:37.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/912: write d4/d7/da0/fb4 [315402,104850] 0 2024-03-11T19:56:37.743 INFO:tasks.workunit.client.0.smithi107.stdout:9/943: mknod d4/d1d/d1f/d78/d105/c145 0 2024-03-11T19:56:37.743 INFO:tasks.workunit.client.0.smithi107.stdout:9/944: fdatasync d4/d9/d9f/dd8/f131 0 2024-03-11T19:56:37.743 INFO:tasks.workunit.client.0.smithi107.stdout:9/945: chown d4/d1d/d1f/d28/fc1 95643 1 2024-03-11T19:56:37.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/913: link d4/d7/d2f/d3b/d101/d24/d26/d29/f3f d4/d7/d2f/d33/d7b/f11f 0 2024-03-11T19:56:37.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/946: symlink d4/d106/l146 0 2024-03-11T19:56:37.745 INFO:tasks.workunit.client.0.smithi107.stdout:4/914: write d4/d7/d2f/d3c/f81 [1253404,82278] 0 2024-03-11T19:56:37.745 INFO:tasks.workunit.client.0.smithi107.stdout:4/915: chown d4/d7/d2f/d3b/d101/d24/d26 63630993 1 2024-03-11T19:56:37.748 INFO:tasks.workunit.client.0.smithi107.stdout:4/916: link d4/d7/d2f/d33/d7b/f106 d4/d7/d2f/d33/d7b/dca/f120 0 2024-03-11T19:56:37.750 INFO:tasks.workunit.client.0.smithi107.stdout:4/917: mknod d4/d7/d2f/d3b/d101/d19/d75/d8e/da5/c121 0 2024-03-11T19:56:37.757 INFO:tasks.workunit.client.0.smithi107.stdout:1/975: dread d6/d7e/d81/d27/d2c/d32/d58/fcc [0,4194304] 0 2024-03-11T19:56:37.762 INFO:tasks.workunit.client.0.smithi107.stdout:2/980: dwrite d1/d1a/d1e/d103/f11f [0,4194304] 0 2024-03-11T19:56:37.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/981: creat d1/d1a/d34/d4f/db3/f14e x:0 0 0 2024-03-11T19:56:37.766 INFO:tasks.workunit.client.0.smithi107.stdout:6/974: dwrite d10/f14 [0,4194304] 0 2024-03-11T19:56:37.769 INFO:tasks.workunit.client.0.smithi107.stdout:6/975: symlink d10/d16/d24/l13a 0 2024-03-11T19:56:37.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/976: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/f14f x:0 0 0 2024-03-11T19:56:37.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/977: write d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/d107/ffb [607927,15460] 0 2024-03-11T19:56:37.777 INFO:tasks.workunit.client.0.smithi107.stdout:2/982: stat d1/d38/d75/ce6 0 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/983: getdents d1/d1a/d8e/d134 0 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/984: chown d1/d38/d75/dd7/l11e 0 1 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/978: rmdir d6/d7e/d81/d27/dfa 39 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/979: dread - d6/f68 zero size 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/980: chown d6/d7e/d81/d27/d2c/d32/dc1/dc9/f11a 382 1 2024-03-11T19:56:37.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/981: creat d6/d118/d14c/f150 x:0 0 0 2024-03-11T19:56:37.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/982: creat d6/d118/f151 x:0 0 0 2024-03-11T19:56:37.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/976: creat d10/d16/d1f/d51/d7a/dba/f13b x:0 0 0 2024-03-11T19:56:37.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/977: fdatasync d10/d16/df6/f111 0 2024-03-11T19:56:37.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/983: creat d6/d7e/d81/d27/d2c/d32/dc1/dc9/de7/f152 x:0 0 0 2024-03-11T19:56:37.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/984: chown d6/d7e/d81/d27/d2c/d32/dc1/dc9/def/f13d 2116 1 2024-03-11T19:56:37.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/978: truncate d10/d19/d3b/ddb/d138/f128 67711 0 2024-03-11T19:56:37.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/947: dwrite d4/f58 [0,4194304] 0 2024-03-11T19:56:37.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/948: getdents d4/d1c 0 2024-03-11T19:56:37.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/979: creat d10/d16/d2b/d39/d6a/d6e/def/db1/f13c x:0 0 0 2024-03-11T19:56:37.795 INFO:tasks.workunit.client.0.smithi107.stdout:9/949: truncate d4/d1d/d1f/d28/d2c/fa5 439461 0 2024-03-11T19:56:37.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/918: dwrite d4/d7/d2f/d3b/d101/d19/d75/d8e/f107 [0,4194304] 0 2024-03-11T19:56:37.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/919: creat d4/d7/d2f/d3b/d101/d24/f122 x:0 0 0 2024-03-11T19:56:37.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/920: symlink d4/l123 0 2024-03-11T19:56:37.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/985: dread d1/d1d/d64/d97/dd1/ff3 [0,4194304] 0 2024-03-11T19:56:37.810 INFO:tasks.workunit.client.0.smithi107.stdout:1/985: write d6/d7e/d81/d27/f3e [1830148,71346] 0 2024-03-11T19:56:37.810 INFO:tasks.workunit.client.0.smithi107.stdout:1/986: chown d6/d7e/da5/dc7/ff8 421882 1 2024-03-11T19:56:37.819 INFO:tasks.workunit.client.0.smithi107.stdout:6/980: dread d10/d16/d2b/f59 [0,4194304] 0 2024-03-11T19:56:37.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/986: dread d1/d38/fc8 [0,4194304] 0 2024-03-11T19:56:37.821 INFO:tasks.workunit.client.0.smithi107.stdout:6/981: mknod d10/d16/d1f/d51/d7a/c13d 0 2024-03-11T19:56:37.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/987: dread d6/d86/dc0/f102 [0,4194304] 0 2024-03-11T19:56:37.827 INFO:tasks.workunit.client.0.smithi107.stdout:4/921: write d4/d7/d2f/d3b/d101/d24/d26/d29/f30 [1039886,67148] 0 2024-03-11T19:56:37.827 INFO:tasks.workunit.client.0.smithi107.stdout:4/922: chown d4/d7/d2f/d3b/d101/d24/d26/d29/f11c 2704031 1 2024-03-11T19:56:37.849 INFO:tasks.workunit.client.0.smithi107.stdout:6/982: write d10/d16/d2b/d39/fb3 [1428767,111507] 0 2024-03-11T19:56:37.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/923: dread d4/fbe [0,4194304] 0 2024-03-11T19:56:37.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/924: symlink d4/d7/d2f/d3b/d101/d24/d26/d119/l124 0 2024-03-11T19:56:37.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/925: write d4/d7/d2f/d3b/d101/d24/d26/ff5 [743316,25556] 0 2024-03-11T19:56:37.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/926: chown d4/d7/d1f 181 1 2024-03-11T19:56:37.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/927: chown d4/d7/da0/f117 7278200 1 2024-03-11T19:56:37.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/928: write d4/d7/d2f/d3c/ffa [1884267,45799] 0 2024-03-11T19:56:37.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/929: readlink d4/d7/d2f/d33/l7f 0 2024-03-11T19:56:37.874 INFO:tasks.workunit.client.0.smithi107.stdout:4/930: mkdir d4/d7/d2f/d3b/d125 0 2024-03-11T19:56:37.875 INFO:tasks.workunit.client.0.smithi107.stdout:4/931: symlink d4/d7/d2f/d3b/d101/d24/d26/d29/dc5/l126 0 2024-03-11T19:56:37.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/932: rmdir d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79 39 2024-03-11T19:56:37.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/933: creat d4/d7/d2f/d3b/d101/d24/d26/f127 x:0 0 0 2024-03-11T19:56:37.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/934: dread d4/fab [0,4194304] 0 2024-03-11T19:56:37.902 INFO:tasks.workunit.client.0.smithi107.stdout:4/935: mkdir d4/d7/d128 0 2024-03-11T19:56:37.924 INFO:tasks.workunit.client.0.smithi107.stdout:4/936: dread d4/fab [4194304,4194304] 0 2024-03-11T19:56:37.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/937: dread d4/d7/d2f/f68 [0,4194304] 0 2024-03-11T19:56:38.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:37 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:56:38.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:37 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:56:38.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:37 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:56:39.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:38 smithi122 ceph-mon[36397]: pgmap v99: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 234 op/s 2024-03-11T19:56:39.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:38 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:56:39.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:38 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:56:39.010 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:38 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:56:39.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:38 smithi145 ceph-mon[36730]: pgmap v99: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 234 op/s 2024-03-11T19:56:39.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:38 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:56:39.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:38 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:56:39.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:38 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:56:39.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:38 smithi107 ceph-mon[31712]: pgmap v99: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 234 op/s 2024-03-11T19:56:39.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:38 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:56:39.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:38 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:56:39.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:38 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:56:41.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:40 smithi122 ceph-mon[36397]: pgmap v100: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 217 op/s 2024-03-11T19:56:41.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:40 smithi145 ceph-mon[36730]: pgmap v100: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 217 op/s 2024-03-11T19:56:41.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:40 smithi107 ceph-mon[31712]: pgmap v100: 129 pgs: 129 active+clean; 2.4 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 58 MiB/s wr, 217 op/s 2024-03-11T19:56:43.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:42 smithi122 ceph-mon[36397]: pgmap v101: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 72 MiB/s wr, 238 op/s 2024-03-11T19:56:43.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:42 smithi145 ceph-mon[36730]: pgmap v101: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 72 MiB/s wr, 238 op/s 2024-03-11T19:56:43.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:42 smithi107 ceph-mon[31712]: pgmap v101: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 72 MiB/s wr, 238 op/s 2024-03-11T19:56:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 -- 172.21.15.107:0/2191294976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c1035d0 msgr2=0x7fd72c1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 --2- 172.21.15.107:0/2191294976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c1035d0 0x7fd72c1039b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd714009930 tx=0x7fd71402bc30 comp rx=0 tx=0).stop 2024-03-11T19:56:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 -- 172.21.15.107:0/2191294976 shutdown_connections 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 --2- 172.21.15.107:0/2191294976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1086a0 0x7fd72c110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 --2- 172.21.15.107:0/2191294976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c103f80 0x7fd72c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 --2- 172.21.15.107:0/2191294976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c1035d0 0x7fd72c1039b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 -- 172.21.15.107:0/2191294976 >> 172.21.15.107:0/2191294976 conn(0x7fd72c0fee30 msgr2=0x7fd72c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 -- 172.21.15.107:0/2191294976 shutdown_connections 2024-03-11T19:56:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.821+0000 7fd733c00640 1 -- 172.21.15.107:0/2191294976 wait complete. 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.822+0000 7fd733c00640 1 Processor -- start 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.822+0000 7fd733c00640 1 -- start start 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.822+0000 7fd733c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 0x7fd72c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd733c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 0x7fd72c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd733c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 0x7fd72c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd733c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c19a2b0 con 0x7fd72c199800 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd733c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c19a420 con 0x7fd72c1035d0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd733c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c19a590 con 0x7fd72c1086a0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 0x7fd72c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 0x7fd72c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 0x7fd72c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 0x7fd72c19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35292/0 (socket says 172.21.15.107:35292) 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd723e00640 1 -- 172.21.15.107:0/2213535465 learned_addr learned my addr 172.21.15.107:0/2213535465 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 -- 172.21.15.107:0/2213535465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 msgr2=0x7fd72c19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 0x7fd72c19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 -- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 msgr2=0x7fd72c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 0x7fd72c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731200640 1 -- 172.21.15.107:0/2213535465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724009840 con 0x7fd72c1035d0 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.823+0000 7fd731c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 0x7fd72c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd731200640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 0x7fd72c19f640 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd714009230 tx=0x7fd71402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd72a000640 1 -- 172.21.15.107:0/2213535465 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140042d0 con 0x7fd72c1035d0 2024-03-11T19:56:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd72a000640 1 -- 172.21.15.107:0/2213535465 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd714004430 con 0x7fd72c1035d0 2024-03-11T19:56:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd72a000640 1 -- 172.21.15.107:0/2213535465 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714003af0 con 0x7fd72c1035d0 2024-03-11T19:56:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd714009560 con 0x7fd72c1035d0 2024-03-11T19:56:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.824+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c1a67e0 con 0x7fd72c1035d0 2024-03-11T19:56:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.827+0000 7fd72a000640 1 -- 172.21.15.107:0/2213535465 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd714005360 con 0x7fd72c1035d0 2024-03-11T19:56:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.827+0000 7fd736835740 1 Processor -- start 2024-03-11T19:56:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 -- start start 2024-03-11T19:56:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb17b620 con 0x5557eb1bfdb0 2024-03-11T19:56:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb12d8c0 con 0x5557eb1c0b20 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd736835740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb12adf0 con 0x5557eb1c9330 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50224/0 (socket says 172.21.15.107:50224) 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd731200640 1 -- 172.21.15.107:0/540418735 learned_addr learned my addr 172.21.15.107:0/540418735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd731c00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.828+0000 7fd723e00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 -- 172.21.15.107:0/540418735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 msgr2=0x5557eb1cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 -- 172.21.15.107:0/540418735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 msgr2=0x5557eb1c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 -- 172.21.15.107:0/540418735 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557eb0a7b00 con 0x5557eb1c0b20 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd723e00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd72a000640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 0x7fd70c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd72a000640 1 -- 172.21.15.107:0/2213535465 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd7140ef9c0 con 0x7fd72c1035d0 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731200640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 0x7fd70c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd731c00640 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1bf870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd72400e860 tx=0x7fd72400ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=407ea5a573609972 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd722000640 1 -- 172.21.15.107:0/540418735 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004d20 con 0x5557eb1c0b20 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd722000640 1 -- 172.21.15.107:0/540418735 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd724003970 con 0x5557eb1c0b20 2024-03-11T19:56:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.829+0000 7fd722000640 1 -- 172.21.15.107:0/540418735 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7240045c0 con 0x5557eb1c0b20 2024-03-11T19:56:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 -- 172.21.15.107:0/540418735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 msgr2=0x5557eb1bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd731200640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 0x7fd70c0a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7140efc20 tx=0x7fd7140dc040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1bf870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd72400e860 tx=0x7fd72400ed20 comp rx=0 tx=0).stop 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 -- 172.21.15.107:0/540418735 shutdown_connections 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 --2- 172.21.15.107:0/540418735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1bf870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 -- 172.21.15.107:0/540418735 >> 172.21.15.107:0/540418735 conn(0x5557eb1874b0 msgr2=0x5557eb1baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 -- 172.21.15.107:0/540418735 shutdown_connections 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.830+0000 7fd736835740 1 -- 172.21.15.107:0/540418735 wait complete. 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 -- start start 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 Processor -- start 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb1265d0 con 0x5557eb1bfdb0 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb0cbb50 con 0x5557eb1c9330 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb0cb7f0 con 0x5557eb1c0b20 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.831+0000 7fd736835740 10 client.0 _ll_register_callbacks cb 0x5557eb0df550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35310/0 (socket says 172.21.15.107:35310) 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50238/0 (socket says 172.21.15.107:50238) 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 -- 172.21.15.107:0/3343713845 learned_addr learned my addr 172.21.15.107:0/3343713845 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd731c00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 -- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 msgr2=0x5557eb1c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 -- 172.21.15.107:0/3343713845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 msgr2=0x5557eb1c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 -- 172.21.15.107:0/3343713845 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72400e540 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd723e00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0126d0 tx=0x7fd71c012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c799cf19677a33a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.832+0000 7fd71aa00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c017070 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd71c00f920 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557eb0f8cd0 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd71c00fae0 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 10 client.25759 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5557eb124c50 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5557eb0f8cd0 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd71c010d10 con 0x5557eb1c0b20 2024-03-11T19:56:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 10 client.25759 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 10 client.25759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 20 client.25759 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb1d94e0 0x5557eb1db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 4 client.25759 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.833+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557eb11cce0 con 0x5557eb1d94e0 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.834+0000 7fd731c00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb1d94e0 0x5557eb1db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.834+0000 7fd731c00640 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb1d94e0 0x5557eb1db900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a6b998b32c6018 server_cookie=24d559a3528afb32 in_seq=0 out_seq=0 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.834+0000 7fd71aa00640 10 client.25759 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd71aa00640 1 -- 172.21.15.107:0/3343713845 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557eb11cce0 con 0x5557eb1d94e0 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd71aa00640 10 client.25759 handle_command_reply: tid=0 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 client.25759 shutdown 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 msgr2=0x5557eb1c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0126d0 tx=0x7fd71c012b90 comp rx=0 tx=0).stop 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 shutdown_connections 2024-03-11T19:56:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb1d94e0 0x5557eb1db900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c6920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 --2- 172.21.15.107:0/3343713845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd71aa00640 0 client.25759 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 >> 172.21.15.107:0/3343713845 conn(0x5557eb1874b0 msgr2=0x5557eb1bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.835+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 shutdown_connections 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.836+0000 7fd736835740 1 -- 172.21.15.107:0/3343713845 wait complete. 2024-03-11T19:56:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.836+0000 7fd736835740 20 client.25759 trim_cache size 0 max 16384 2024-03-11T19:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 Processor -- start 2024-03-11T19:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 -- start start 2024-03-11T19:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eae6ee60 con 0x5557eb1bfdb0 2024-03-11T19:56:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb1b67d0 con 0x5557eb1c9330 2024-03-11T19:56:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd736835740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb1d7c00 con 0x5557eb1c0b20 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:35314/0 (socket says 172.21.15.107:35314) 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.850+0000 7fd723e00640 1 -- 172.21.15.107:0/3533870016 learned_addr learned my addr 172.21.15.107:0/3533870016 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd731c00640 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 -- 172.21.15.107:0/3533870016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 msgr2=0x5557eb1c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 -- 172.21.15.107:0/3533870016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 msgr2=0x5557eb1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 -- 172.21.15.107:0/3533870016 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557eb0c1530 con 0x5557eb1c0b20 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd731c00640 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd723e00640 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd71c01c4f0 tx=0x7fd71c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c151a7ecaa9b8b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd721600640 1 -- 172.21.15.107:0/3533870016 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c017040 con 0x5557eb1c0b20 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd721600640 1 -- 172.21.15.107:0/3533870016 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd71c01cc40 con 0x5557eb1c0b20 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.851+0000 7fd721600640 1 -- 172.21.15.107:0/3533870016 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0054e0 con 0x5557eb1c0b20 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- 172.21.15.107:0/3533870016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 msgr2=0x5557eb1c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd71c01c4f0 tx=0x7fd71c009740 comp rx=0 tx=0).stop 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- 172.21.15.107:0/3533870016 shutdown_connections 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c9330 0x5557eb1c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c0b20 0x5557eb1c4cb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 --2- 172.21.15.107:0/3533870016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- 172.21.15.107:0/3533870016 >> 172.21.15.107:0/3533870016 conn(0x5557eb1874b0 msgr2=0x5557eb1ba5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- 172.21.15.107:0/3533870016 shutdown_connections 2024-03-11T19:56:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- 172.21.15.107:0/3533870016 wait complete. 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.852+0000 7fd736835740 1 -- start start 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 Processor -- start 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb22fb10 con 0x5557eb1bfdb0 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb22fc80 con 0x5557eb1c0b20 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb22fe40 con 0x5557eb1c9330 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd736835740 10 client.0 _ll_register_callbacks cb 0x7fd71c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd723e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd731c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50264/0 (socket says 172.21.15.107:50264) 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.853+0000 7fd731c00640 1 -- 172.21.15.107:0/2947061076 learned_addr learned my addr 172.21.15.107:0/2947061076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 -- 172.21.15.107:0/2947061076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 msgr2=0x5557eb1cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 -- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 msgr2=0x5557eb1cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 -- 172.21.15.107:0/2947061076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557eb124c50 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd723e00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd731c00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd724000f80 tx=0x7fd72402f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8000eb51792783 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd71aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004470 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd724011e10 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.854+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557eb14bd30 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7240045d0 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 10 client.15636 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5557eb0e72c0 con 0x5557eb1bfdb0 2024-03-11T19:56:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5557eb14bd30 con 0x5557eb1bfdb0 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd724025040 con 0x5557eb1bfdb0 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 20 client.15636 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb2315d0 0x5557eb2339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 4 client.15636 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557eae26600 con 0x5557eb2315d0 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.855+0000 7fd723e00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb2315d0 0x5557eb2339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.856+0000 7fd71aa00640 10 client.15636 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:44.856+0000 7fd723e00640 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb2315d0 0x5557eb2339f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b9d235484993ef server_cookie=836f675d801b0be4 in_seq=0 out_seq=0 2024-03-11T19:56:44.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/829: chown de/d11/d74/da2/dd3/dd4/ca6 476 1 2024-03-11T19:56:44.870 INFO:tasks.workunit.client.0.smithi107.stdout:0/830: mknod de/d11/d23/d24/d4f/d89/cfe 0 2024-03-11T19:56:44.870 INFO:tasks.workunit.client.0.smithi107.stdout:0/831: dread - de/d11/d23/d24/ff6 zero size 2024-03-11T19:56:44.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/832: dread de/d11/d23/f76 [0,4194304] 0 2024-03-11T19:56:44.888 INFO:tasks.workunit.client.0.smithi107.stdout:0/833: creat de/d11/d23/d32/d34/d39/d55/d7c/dd7/fff x:0 0 0 2024-03-11T19:56:44.900 INFO:tasks.workunit.client.0.smithi107.stdout:0/834: truncate de/d11/d74/f79 554048 0 2024-03-11T19:56:44.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/835: dwrite de/d11/d23/d32/d34/d39/d55/d7c/def/ff0 [0,4194304] 0 2024-03-11T19:56:44.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/836: creat de/d9e/f100 x:0 0 0 2024-03-11T19:56:44.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/837: chown de/d11/f91 2350619 1 2024-03-11T19:56:45.007 INFO:tasks.workunit.client.0.smithi107.stdout:0/838: read de/d11/fb0 [4098997,13455] 0 2024-03-11T19:56:45.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:44 smithi122 ceph-mon[36397]: pgmap v102: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 75 MiB/s wr, 260 op/s 2024-03-11T19:56:45.009 INFO:tasks.workunit.client.0.smithi107.stdout:0/839: mkdir de/d11/d23/d32/d34/d39/d55/d7c/def/d101 0 2024-03-11T19:56:45.009 INFO:tasks.workunit.client.0.smithi107.stdout:0/840: chown de/d7a/f85 46655009 1 2024-03-11T19:56:45.010 INFO:tasks.workunit.client.0.smithi107.stdout:0/841: chown de/d11/d74/da2/dd3/dd4/dbf/dd1/fd6 234822677 1 2024-03-11T19:56:45.013 INFO:tasks.workunit.client.0.smithi107.stdout:0/842: truncate de/d7a/f85 382922 0 2024-03-11T19:56:45.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:44 smithi145 ceph-mon[36730]: pgmap v102: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 75 MiB/s wr, 260 op/s 2024-03-11T19:56:45.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:44 smithi107 ceph-mon[31712]: pgmap v102: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 75 MiB/s wr, 260 op/s 2024-03-11T19:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd71aa00640 1 -- 172.21.15.107:0/2947061076 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557eae26600 con 0x5557eb2315d0 2024-03-11T19:56:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd71aa00640 10 client.15636 handle_command_reply: tid=0 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 client.15636 shutdown 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 msgr2=0x5557eb1ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd724000f80 tx=0x7fd72402f040 comp rx=0 tx=0).stop 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 shutdown_connections 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5557eb2315d0 0x5557eb2339f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557eb1c0b20 0x5557eb1cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5557eb1c9330 0x5557eb1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 --2- 172.21.15.107:0/2947061076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557eb1bfdb0 0x5557eb1ced30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 >> 172.21.15.107:0/2947061076 conn(0x5557eb1874b0 msgr2=0x5557eb1bf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd71aa00640 0 client.15636 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 shutdown_connections 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 1 -- 172.21.15.107:0/2947061076 wait complete. 2024-03-11T19:56:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.764+0000 7fd736835740 20 client.15636 trim_cache size 0 max 16384 2024-03-11T19:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:56:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 msgr2=0x7fd70c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 0x7fd70c0a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7140efc20 tx=0x7fd7140dc040 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 msgr2=0x7fd72c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 0x7fd72c19f640 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd714009230 tx=0x7fd71402c5d0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 shutdown_connections 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd70c0a1cc0 0x7fd70c0a4180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd72c199800 0x7fd72c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd72c1086a0 0x7fd72c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 --2- 172.21.15.107:0/2213535465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd72c1035d0 0x7fd72c19f640 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.765+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 >> 172.21.15.107:0/2213535465 conn(0x7fd72c0fee30 msgr2=0x7fd72c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.766+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 shutdown_connections 2024-03-11T19:56:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:46.766+0000 7fd733c00640 1 -- 172.21.15.107:0/2213535465 wait complete. 2024-03-11T19:56:47.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:46 smithi122 ceph-mon[36397]: pgmap v103: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 74 MiB/s wr, 239 op/s 2024-03-11T19:56:47.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:46 smithi145 ceph-mon[36730]: pgmap v103: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 74 MiB/s wr, 239 op/s 2024-03-11T19:56:47.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:46 smithi107 ceph-mon[31712]: pgmap v103: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 74 MiB/s wr, 239 op/s 2024-03-11T19:56:48.256 INFO:tasks.workunit.client.0.smithi107.stdout:6/983: rename d10/d19/f65 to d10/d16/d2b/d39/d6a/d6e/f13e 0 2024-03-11T19:56:48.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/938: rename d4/d7/d2f/d3b/d101/d19/d75/cad to d4/d7/d2f/d3b/d101/d19/c129 0 2024-03-11T19:56:48.284 INFO:tasks.workunit.client.0.smithi107.stdout:0/843: rename de/d11/d23/d32/d34/d39/l5d to de/d11/d23/d24/d83/dc4/l102 0 2024-03-11T19:56:48.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/844: creat de/d11/d23/d32/f103 x:0 0 0 2024-03-11T19:56:48.287 INFO:tasks.workunit.client.0.smithi107.stdout:0/845: truncate de/d11/d23/d24/da4/dae/fd0 608662 0 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/950: sync 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:8/911: sync 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:1/988: sync 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:2/987: sync 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/951: stat d4/d1d/d5c/fa6 0 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:1/989: dread - d6/d7e/d81/d27/d2c/d72/ff0 zero size 2024-03-11T19:56:48.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/984: dwrite d10/d16/d2b/d39/d6a/d6e/def/db8/f10d [0,4194304] 0 2024-03-11T19:56:48.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/912: creat dd/d16/f11b x:0 0 0 2024-03-11T19:56:48.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/913: write dd/de/dfc/d10f/f5b [1449128,114732] 0 2024-03-11T19:56:48.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/988: symlink d1/d1a/d1e/d26/dd6/df4/d12b/d13e/l14f 0 2024-03-11T19:56:48.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/914: truncate dd/d16/f7f 4048952 0 2024-03-11T19:56:48.357 INFO:tasks.workunit.client.0.smithi107.stdout:6/985: mkdir d10/d13f 0 2024-03-11T19:56:48.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/915: creat dd/de/d1b/f11c x:0 0 0 2024-03-11T19:56:48.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/916: stat dd/d13/d18/f5c 0 2024-03-11T19:56:48.363 INFO:tasks.workunit.client.0.smithi107.stdout:2/989: truncate d1/d21/dbb/fd0 2367659 0 2024-03-11T19:56:48.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/939: dwrite d4/d7/d2f/d3c/f77 [0,4194304] 0 2024-03-11T19:56:48.376 INFO:tasks.workunit.client.0.smithi107.stdout:4/940: getdents d4/d7/d1f/d82 0 2024-03-11T19:56:48.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/941: fsync d4/d7/d2f/d3b/d101/d24/d26/d29/f64 0 2024-03-11T19:56:48.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/952: dread d4/d1d/d1f/d28/f6e [4194304,4194304] 0 2024-03-11T19:56:48.381 INFO:tasks.workunit.client.0.smithi107.stdout:0/846: dwrite de/d11/d23/d24/d4f/fc7 [0,4194304] 0 2024-03-11T19:56:48.382 INFO:tasks.workunit.client.0.smithi107.stdout:4/942: mknod d4/d7/d2f/d3b/d101/d24/d26/d29/c12a 0 2024-03-11T19:56:48.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/953: truncate d4/d1d/d5c/d64/fa8 861241 0 2024-03-11T19:56:48.389 INFO:tasks.workunit.client.0.smithi107.stdout:9/954: symlink d4/d9/dae/d6d/l147 0 2024-03-11T19:56:48.390 INFO:tasks.workunit.client.0.smithi107.stdout:9/955: fdatasync d4/d9/d9f/dd8/fe4 0 2024-03-11T19:56:48.402 INFO:tasks.workunit.client.0.smithi107.stdout:1/990: dread d6/d28/f80 [0,4194304] 0 2024-03-11T19:56:48.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/917: rename dd/de/d24/d28/d47/l67 to dd/de/d1b/d20/d5e/dc8/l11d 0 2024-03-11T19:56:48.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/943: dread d4/d7/d2f/d3b/d101/d19/f70 [0,4194304] 0 2024-03-11T19:56:48.413 INFO:tasks.workunit.client.0.smithi107.stdout:0/847: rename de/d11/d74/da2/dd3/dd4/dbf/dd1 to de/d7a/d104 0 2024-03-11T19:56:48.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/944: rename d4/d7/d1f/l87 to d4/d7/d2f/d33/d7b/l12b 0 2024-03-11T19:56:48.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/945: fsync d4/d7/d2f/d3b/d101/f109 0 2024-03-11T19:56:48.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/990: dwrite d1/d1a/d34/f115 [0,4194304] 0 2024-03-11T19:56:48.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/991: write d1/d1a/d1e/d103/d106/f124 [336238,35270] 0 2024-03-11T19:56:48.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/992: creat d1/d1a/d1e/f150 x:0 0 0 2024-03-11T19:56:48.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/956: write d4/d1d/d1f/d28/d2c/f82 [991569,105163] 0 2024-03-11T19:56:48.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/993: symlink d1/d38/d75/l151 0 2024-03-11T19:56:48.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/957: mknod d4/d1d/d5c/d64/db3/d100/c148 0 2024-03-11T19:56:48.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/958: rename d4/d9/dae/d6d/f68 to d4/d9/d4d/f149 0 2024-03-11T19:56:48.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/959: fsync d4/d1d/d4b/f59 0 2024-03-11T19:56:48.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/960: stat d4/d1d/d1f/d28/f67 0 2024-03-11T19:56:48.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/961: unlink d4/d1c/d37/dd9/c10a 0 2024-03-11T19:56:48.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/962: readlink d4/d9/dae/d6d/db0/lb8 0 2024-03-11T19:56:48.434 INFO:tasks.workunit.client.0.smithi107.stdout:9/963: mkdir d4/d1d/d1f/d78/d105/d14a 0 2024-03-11T19:56:48.434 INFO:tasks.workunit.client.0.smithi107.stdout:9/964: rmdir d4/d1d/d1f/d28/d5d/df2 39 2024-03-11T19:56:48.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/965: symlink d4/d1d/d1f/d28/l14b 0 2024-03-11T19:56:48.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/966: mkdir d4/d1d/d1f/d78/d118/d14c 0 2024-03-11T19:56:48.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/967: dread - d4/d1d/d4b/f10f zero size 2024-03-11T19:56:48.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/968: mknod d4/d9/d9f/dd8/c14d 0 2024-03-11T19:56:48.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/969: chown d4/d1d/d5c/d97/d125/l128 11470 1 2024-03-11T19:56:48.440 INFO:tasks.workunit.client.0.smithi107.stdout:9/970: mkdir d4/d106/d14e 0 2024-03-11T19:56:48.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/946: dwrite d4/d7/d2f/d3b/d101/d19/d75/d8e/de9/ffc [0,4194304] 0 2024-03-11T19:56:48.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/848: dread de/f88 [0,4194304] 0 2024-03-11T19:56:48.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/947: creat d4/d7/d2f/d3b/d101/d19/d75/d8e/de9/f12c x:0 0 0 2024-03-11T19:56:48.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/948: fdatasync d4/d7/d2f/d3b/d101/d24/d26/d29/d36/f8c 0 2024-03-11T19:56:48.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/949: dread - d4/d7/db/db8/fc8 zero size 2024-03-11T19:56:48.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/918: dwrite dd/de/d24/f113 [0,4194304] 0 2024-03-11T19:56:48.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/950: dread f0 [0,4194304] 0 2024-03-11T19:56:48.482 INFO:tasks.workunit.client.0.smithi107.stdout:4/951: truncate d4/d7/d2f/d33/d7b/f99 924319 0 2024-03-11T19:56:48.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/994: dwrite d1/d1a/d34/f147 [0,4194304] 0 2024-03-11T19:56:48.486 INFO:tasks.workunit.client.0.smithi107.stdout:2/995: rename d1/fe2 to d1/d1a/d34/d4f/d63/d14a/f152 0 2024-03-11T19:56:48.486 INFO:tasks.workunit.client.0.smithi107.stdout:2/996: creat d1/d21/d35/f153 x:0 0 0 2024-03-11T19:56:48.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/971: dwrite d4/fe [4194304,4194304] 0 2024-03-11T19:56:48.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/972: chown d4/d1c/d37/dd9/de9 999 1 2024-03-11T19:56:48.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/997: dread d1/d21/fa7 [0,4194304] 0 2024-03-11T19:56:48.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/998: dread - d1/d1d/d64/f148 zero size 2024-03-11T19:56:48.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/999: fsync d1/fdd 0 2024-03-11T19:56:48.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/919: dwrite dd/de/d1b/f83 [0,4194304] 0 2024-03-11T19:56:48.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/920: dread - dd/de/d1b/d20/f9e zero size 2024-03-11T19:56:48.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/921: symlink dd/de/d1b/d20/d56/da4/dee/l11e 0 2024-03-11T19:56:48.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/922: fsync dd/d13/f6c 0 2024-03-11T19:56:48.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/923: readlink dd/dac/lbc 0 2024-03-11T19:56:48.526 INFO:tasks.workunit.client.0.smithi107.stdout:8/924: dread dd/de/d1b/ff9 [0,4194304] 0 2024-03-11T19:56:48.529 INFO:tasks.workunit.client.0.smithi107.stdout:9/973: dwrite d4/d1d/d1f/d28/d2c/f5b [4194304,4194304] 0 2024-03-11T19:56:48.529 INFO:tasks.workunit.client.0.smithi107.stdout:9/974: dread - d4/d1c/d37/ff3 zero size 2024-03-11T19:56:48.530 INFO:tasks.workunit.client.0.smithi107.stdout:9/975: mknod d4/d1c/d37/c14f 0 2024-03-11T19:56:48.531 INFO:tasks.workunit.client.0.smithi107.stdout:9/976: rmdir d4/d1d/d5c/d64/db3/d100 39 2024-03-11T19:56:48.533 INFO:tasks.workunit.client.0.smithi107.stdout:9/977: mkdir d4/d1d/d1f/d28/d2c/d96/dd3/d150 0 2024-03-11T19:56:48.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/978: creat d4/d106/d14e/f151 x:0 0 0 2024-03-11T19:56:48.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/979: fdatasync d4/d1d/d5c/fde 0 2024-03-11T19:56:48.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/980: truncate d4/d1c/d37/f50 1240903 0 2024-03-11T19:56:48.537 INFO:tasks.workunit.client.0.smithi107.stdout:9/981: rename d4/d1d/d1f/ff0 to d4/d1d/d5c/f152 0 2024-03-11T19:56:48.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/982: mknod d4/d1d/d1f/d28/d5d/df2/c153 0 2024-03-11T19:56:48.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/983: chown d4/d9/d9f/l98 0 1 2024-03-11T19:56:48.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/984: write d4/d1d/d5c/f13c [892030,111211] 0 2024-03-11T19:56:48.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/985: readlink d4/d9/dae/d6d/db0/lb8 0 2024-03-11T19:56:48.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/952: dwrite d4/d7/d2f/d3b/d101/d93/dc3/dd0/fec [0,4194304] 0 2024-03-11T19:56:48.584 INFO:tasks.workunit.client.0.smithi107.stdout:9/986: dwrite d4/d1d/d5c/f142 [0,4194304] 0 2024-03-11T19:56:48.589 INFO:tasks.workunit.client.0.smithi107.stdout:9/987: creat d4/d106/d12b/f154 x:0 0 0 2024-03-11T19:56:48.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/988: mknod d4/d1c/d37/c155 0 2024-03-11T19:56:48.593 INFO:tasks.workunit.client.0.smithi107.stdout:0/849: dread de/d11/f91 [0,4194304] 0 2024-03-11T19:56:48.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/989: link d4/d1d/d1f/d28/fe7 d4/d9/dae/f156 0 2024-03-11T19:56:48.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/990: write d4/d1d/d1f/d28/d2c/f34 [2494503,82928] 0 2024-03-11T19:56:48.616 INFO:tasks.workunit.client.0.smithi107.stdout:4/953: dread d4/d7/d2f/d3c/f73 [0,4194304] 0 2024-03-11T19:56:48.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/954: rmdir d4/d7/db/db8 39 2024-03-11T19:56:48.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/955: write d4/d7/d2f/d3b/d101/d19/d75/d8e/fdd [4803848,17644] 0 2024-03-11T19:56:48.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/925: dwrite dd/de/f10b [4194304,4194304] 0 2024-03-11T19:56:48.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/926: creat dd/d16/f11f x:0 0 0 2024-03-11T19:56:48.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/991: dread d4/d1d/f3b [0,4194304] 0 2024-03-11T19:56:48.632 INFO:tasks.workunit.client.0.smithi107.stdout:9/992: creat d4/d1c/d37/dd9/de9/df8/f157 x:0 0 0 2024-03-11T19:56:48.633 INFO:tasks.workunit.client.0.smithi107.stdout:9/993: mknod d4/d1d/d1f/d28/d2c/d96/dd3/d150/c158 0 2024-03-11T19:56:48.634 INFO:tasks.workunit.client.0.smithi107.stdout:9/994: creat d4/d9/dae/d6d/f159 x:0 0 0 2024-03-11T19:56:48.635 INFO:tasks.workunit.client.0.smithi107.stdout:9/995: unlink d4/d9/dae/l72 0 2024-03-11T19:56:48.636 INFO:tasks.workunit.client.0.smithi107.stdout:9/996: chown d4/d1d/d1f/l36 1366454400 1 2024-03-11T19:56:48.637 INFO:tasks.workunit.client.0.smithi107.stdout:9/997: rmdir d4/d1c/d37/dd9/de9/df1 39 2024-03-11T19:56:48.813 INFO:tasks.workunit.client.0.smithi107.stdout:8/927: rename dd/d13/d18/d81/ddc to dd/de/d1b/d20/d56/da4/dee/d120 0 2024-03-11T19:56:48.815 INFO:tasks.workunit.client.0.smithi107.stdout:4/956: dwrite d4/d7/db/db8/fc8 [0,4194304] 0 2024-03-11T19:56:48.816 INFO:tasks.workunit.client.0.smithi107.stdout:4/957: truncate d4/d7/d2f/d3b/d101/d24/f6a 262400 0 2024-03-11T19:56:48.819 INFO:tasks.workunit.client.0.smithi107.stdout:4/958: link d4/d7/db/l10 d4/d7/d2f/d3b/d101/d19/l12d 0 2024-03-11T19:56:48.844 INFO:tasks.workunit.client.0.smithi107.stdout:4/959: link d4/d7/d2f/d3b/d101/d19/d75/d8e/fbc d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79/dbb/f12e 0 2024-03-11T19:56:48.854 INFO:tasks.workunit.client.0.smithi107.stdout:8/928: dread dd/f58 [0,4194304] 0 2024-03-11T19:56:48.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/929: stat dd/de/d1b/d20/d56/da4/dee/d120/c63 0 2024-03-11T19:56:48.899 INFO:tasks.workunit.client.0.smithi107.stdout:8/930: dread dd/de/d1b/f83 [0,4194304] 0 2024-03-11T19:56:48.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/960: dwrite d4/d7/d2f/d3b/d101/d93/fe0 [0,4194304] 0 2024-03-11T19:56:48.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/961: stat d4/d7/d2f/d3b/d101/d24/d26/d29/dc5 0 2024-03-11T19:56:48.931 INFO:tasks.workunit.client.0.smithi107.stdout:4/962: creat d4/d7/d2f/d3b/d101/d19/d75/d8e/da5/f12f x:0 0 0 2024-03-11T19:56:48.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/963: link d4/d7/d2f/d3b/d101/d19/d75/d8e/f107 d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79/f130 0 2024-03-11T19:56:48.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/964: creat d4/d7/d2f/d3b/d101/f131 x:0 0 0 2024-03-11T19:56:48.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/965: chown d4/d7/d2f/d3b/d101/d24/d26/l28 22 1 2024-03-11T19:56:48.946 INFO:tasks.workunit.client.0.smithi107.stdout:4/966: getdents d4/d7/d2f/d3b/d101/d24/d26/d29/dc5 0 2024-03-11T19:56:48.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/931: dwrite dd/d13/d18/f1a [0,4194304] 0 2024-03-11T19:56:49.004 INFO:tasks.workunit.client.0.smithi107.stdout:8/932: write dd/de/d1b/d20/d56/da4/dee/d120/de8/d117/ffb [2880380,21103] 0 2024-03-11T19:56:49.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:48 smithi122 ceph-mon[36397]: pgmap v104: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 75 MiB/s wr, 283 op/s 2024-03-11T19:56:49.025 INFO:tasks.workunit.client.0.smithi107.stdout:4/967: dwrite d4/d7/da0/f117 [0,4194304] 0 2024-03-11T19:56:49.025 INFO:tasks.workunit.client.0.smithi107.stdout:4/968: stat d4/d7/db/d5e/fed 0 2024-03-11T19:56:49.034 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:48 smithi145 ceph-mon[36730]: pgmap v104: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 75 MiB/s wr, 283 op/s 2024-03-11T19:56:49.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/998: sync 2024-03-11T19:56:49.037 INFO:tasks.workunit.client.0.smithi107.stdout:0/850: sync 2024-03-11T19:56:49.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/991: sync 2024-03-11T19:56:49.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/986: sync 2024-03-11T19:56:49.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/987: fdatasync d10/d19/d3b/f5e 0 2024-03-11T19:56:49.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/988: mknod d10/d16/d1f/d51/d7a/dba/c140 0 2024-03-11T19:56:49.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/989: dread - d10/d16/d2b/d39/fcf zero size 2024-03-11T19:56:49.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:48 smithi107 ceph-mon[31712]: pgmap v104: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 75 MiB/s wr, 283 op/s 2024-03-11T19:56:49.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/851: truncate f5 1871940 0 2024-03-11T19:56:49.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/852: chown de/f18 354 1 2024-03-11T19:56:49.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/992: symlink d6/d28/dc4/l153 0 2024-03-11T19:56:49.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/993: dread - d6/d5b/f142 zero size 2024-03-11T19:56:49.043 INFO:tasks.workunit.client.0.smithi107.stdout:0/853: unlink de/f16 0 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/990: rmdir d10/d16/d2b/d39/d6a/d6e 39 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:1/994: rename d6/d7e/d81/d27/d2c/d72/dc3/dd4/d62/c10e to d6/d7e/da5/dc7/c154 0 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/991: readlink d10/d16/d2b/d39/d6a/lfa 0 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:0/854: symlink de/d11/d23/d24/da4/l105 0 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:0/855: write de/d11/d23/d32/d34/d39/d3a/f50 [4388215,40410] 0 2024-03-11T19:56:49.045 INFO:tasks.workunit.client.0.smithi107.stdout:1/995: creat d6/d7e/d81/d27/d2c/d72/dc3/dd4/f155 x:0 0 0 2024-03-11T19:56:49.048 INFO:tasks.workunit.client.0.smithi107.stdout:0/856: truncate de/d58/fa7 1831907 0 2024-03-11T19:56:49.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/857: rmdir de/d11/de8 0 2024-03-11T19:56:49.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/858: readlink de/d11/d23/d32/d34/d39/d55/d7c/ldf 0 2024-03-11T19:56:49.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/859: write de/d11/d23/d32/faf [166916,18485] 0 2024-03-11T19:56:49.055 INFO:tasks.workunit.client.0.smithi107.stdout:8/933: write dd/de/d1b/f9a [2525394,87779] 0 2024-03-11T19:56:49.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/934: dread - dd/de/d1b/d20/d56/d87/ffe zero size 2024-03-11T19:56:49.058 INFO:tasks.workunit.client.0.smithi107.stdout:6/992: creat d10/d16/d2b/d39/d6a/d6e/def/f141 x:0 0 0 2024-03-11T19:56:49.061 INFO:tasks.workunit.client.0.smithi107.stdout:6/993: creat d10/d19/d11c/f142 x:0 0 0 2024-03-11T19:56:49.065 INFO:tasks.workunit.client.0.smithi107.stdout:6/994: mknod d10/d16/d2b/d8d/c143 0 2024-03-11T19:56:49.069 INFO:tasks.workunit.client.0.smithi107.stdout:6/995: symlink d10/d16/d2b/d39/d6a/d6e/l144 0 2024-03-11T19:56:49.072 INFO:tasks.workunit.client.0.smithi107.stdout:6/996: creat d10/d16/df6/f145 x:0 0 0 2024-03-11T19:56:49.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/996: read d6/f15 [2884278,10111] 0 2024-03-11T19:56:49.085 INFO:tasks.workunit.client.0.smithi107.stdout:4/969: write d4/d7/d2f/d3b/d101/d24/f3e [3993927,120729] 0 2024-03-11T19:56:49.086 INFO:tasks.workunit.client.0.smithi107.stdout:4/970: write d4/d7/d2f/d3b/d101/d24/feb [1003859,50693] 0 2024-03-11T19:56:49.086 INFO:tasks.workunit.client.0.smithi107.stdout:4/971: fdatasync d4/d7/d1f/f102 0 2024-03-11T19:56:49.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/997: symlink d6/d7e/d81/d27/d2c/d7d/dd8/l156 0 2024-03-11T19:56:49.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/998: chown d6/f1d 3849 1 2024-03-11T19:56:49.090 INFO:tasks.workunit.client.0.smithi107.stdout:9/999: dread d4/d1c/d37/fbb [4194304,4194304] 0 2024-03-11T19:56:49.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/999: symlink d6/d86/dc0/l157 0 2024-03-11T19:56:49.101 INFO:tasks.workunit.client.0.smithi107.stdout:0/860: rename de/d11/d74/ce7 to de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda/c106 0 2024-03-11T19:56:49.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/997: dread d10/d19/d3b/f8b [0,4194304] 0 2024-03-11T19:56:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/861: creat de/d11/d23/d24/d83/dc4/f107 x:0 0 0 2024-03-11T19:56:49.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/862: chown de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda 1151659 1 2024-03-11T19:56:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/998: getdents d10/d16/d24/d2e/ded 0 2024-03-11T19:56:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:8/935: dwrite dd/d13/d18/d81/f8d [0,4194304] 0 2024-03-11T19:56:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:8/936: readlink dd/de/d1b/d20/d5e/dc8/l11d 0 2024-03-11T19:56:49.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/863: symlink de/d11/d23/d24/d83/dc4/de4/l108 0 2024-03-11T19:56:49.112 INFO:tasks.workunit.client.0.smithi107.stdout:0/864: write de/f44 [4365834,44175] 0 2024-03-11T19:56:49.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/865: mkdir de/d11/d23/d24/da4/dae/dcd/d109 0 2024-03-11T19:56:49.118 INFO:tasks.workunit.client.0.smithi107.stdout:0/866: fsync de/d11/f35 0 2024-03-11T19:56:49.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/937: rename dd/de/d24/d28 to dd/de/d1b/d20/d56/da4/dee/d120/d90/d121 0 2024-03-11T19:56:49.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/867: symlink de/d9e/dc8/l10a 0 2024-03-11T19:56:49.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/868: fdatasync de/d11/d23/d32/d34/f6c 0 2024-03-11T19:56:49.124 INFO:tasks.workunit.client.0.smithi107.stdout:0/869: getdents de/d11/d74/da2/dd3/dd4/dbf 0 2024-03-11T19:56:49.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/972: write d4/d7/d2f/d3b/d101/d19/fa7 [675592,33594] 0 2024-03-11T19:56:49.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/973: chown d4/d7/d2f/d33/d7b/f106 3011 1 2024-03-11T19:56:49.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/974: rename d4/d7/db/d5c/l72 to d4/d7/d2f/d3b/d101/d19/l132 0 2024-03-11T19:56:49.132 INFO:tasks.workunit.client.0.smithi107.stdout:4/975: symlink d4/d7/d2f/d3b/d101/d24/d26/l133 0 2024-03-11T19:56:49.132 INFO:tasks.workunit.client.0.smithi107.stdout:4/976: creat d4/d7/d2f/d3b/d101/d24/f134 x:0 0 0 2024-03-11T19:56:49.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/977: symlink d4/d7/d2f/d3b/d101/l135 0 2024-03-11T19:56:49.138 INFO:tasks.workunit.client.0.smithi107.stdout:4/978: write d4/d7/d1f/f52 [225438,94599] 0 2024-03-11T19:56:49.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/979: getdents d4/d7/d2f/d33/d7b 0 2024-03-11T19:56:49.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/980: chown d4/d7/db/d5c/lef 6 1 2024-03-11T19:56:49.156 INFO:tasks.workunit.client.0.smithi107.stdout:6/999: dwrite d10/f129 [0,4194304] 0 2024-03-11T19:56:49.176 INFO:tasks.workunit.client.0.smithi107.stdout:0/870: dwrite de/d11/d23/d24/da4/fb3 [0,4194304] 0 2024-03-11T19:56:49.178 INFO:tasks.workunit.client.0.smithi107.stdout:0/871: mkdir de/d11/d74/da2/dd3/d10b 0 2024-03-11T19:56:49.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/872: unlink de/d11/d23/c37 0 2024-03-11T19:56:49.180 INFO:tasks.workunit.client.0.smithi107.stdout:0/873: chown de/c1d 201403 1 2024-03-11T19:56:49.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/981: dwrite d4/d7/d2f/d3b/d101/d24/d26/d119/fcd [0,4194304] 0 2024-03-11T19:56:49.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/982: truncate d4/d7/da0/f117 4357010 0 2024-03-11T19:56:49.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/983: write d4/d7/d2f/d3b/d101/d19/f2a [2606873,44958] 0 2024-03-11T19:56:49.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/984: readlink d4/d7/d2f/d3b/d101/ldb 0 2024-03-11T19:56:49.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/874: write de/d11/d23/d24/da4/fbd [1804266,33423] 0 2024-03-11T19:56:49.203 INFO:tasks.workunit.client.0.smithi107.stdout:4/985: dread d4/d7/d2f/d3b/d101/d19/d75/fb5 [0,4194304] 0 2024-03-11T19:56:49.203 INFO:tasks.workunit.client.0.smithi107.stdout:4/986: write d4/d7/d2f/d3b/d101/d19/d75/d8e/fa8 [1765493,92721] 0 2024-03-11T19:56:49.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/987: dread d4/d7/d2f/d3b/d101/d19/fa7 [0,4194304] 0 2024-03-11T19:56:49.224 INFO:tasks.workunit.client.0.smithi107.stdout:0/875: dwrite de/d11/fb0 [0,4194304] 0 2024-03-11T19:56:49.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/988: dread d4/d7/db/d5e/fcc [0,4194304] 0 2024-03-11T19:56:49.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/989: truncate d4/d7/d2f/d3b/d101/d24/f5b 1040598 0 2024-03-11T19:56:49.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/990: mknod d4/d7/d2f/d3b/d101/d93/dc3/dd0/d79/c136 0 2024-03-11T19:56:49.228 INFO:tasks.workunit.client.0.smithi107.stdout:0/876: write de/d11/d23/d32/d34/d39/f6e [156072,76106] 0 2024-03-11T19:56:49.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/877: creat de/d11/d23/d24/d4f/f10c x:0 0 0 2024-03-11T19:56:49.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/878: write de/d11/f51 [4691828,82214] 0 2024-03-11T19:56:49.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/879: fdatasync de/d11/d23/f9d 0 2024-03-11T19:56:49.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/991: write d4/d7/da0/fb4 [1287171,76920] 0 2024-03-11T19:56:49.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/938: sync 2024-03-11T19:56:49.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/939: fdatasync dd/de/d24/f97 0 2024-03-11T19:56:49.249 INFO:tasks.workunit.client.0.smithi107.stdout:8/940: read dd/f69 [1396809,96025] 0 2024-03-11T19:56:49.250 INFO:tasks.workunit.client.0.smithi107.stdout:8/941: chown dd/d13/d32/f78 1231 1 2024-03-11T19:56:49.250 INFO:tasks.workunit.client.0.smithi107.stdout:8/942: creat dd/de/d1b/d20/d56/da4/dee/d120/f122 x:0 0 0 2024-03-11T19:56:49.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/943: mkdir dd/de/d1b/d20/d5e/dc8/d123 0 2024-03-11T19:56:49.252 INFO:tasks.workunit.client.0.smithi107.stdout:8/944: chown dd/f58 3329268 1 2024-03-11T19:56:49.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/945: mkdir dd/de/d1b/d20/d56/da4/dee/d120/d90/d124 0 2024-03-11T19:56:49.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/946: mkdir dd/d13/d125 0 2024-03-11T19:56:49.283 INFO:tasks.workunit.client.0.smithi107.stdout:8/947: dwrite dd/de/dfc/d10f/d7e/fcb [0,4194304] 0 2024-03-11T19:56:49.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/948: getdents dd/d16 0 2024-03-11T19:56:49.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/949: dread dd/de/dfc/d10f/d7e/fae [0,4194304] 0 2024-03-11T19:56:49.320 INFO:tasks.workunit.client.0.smithi107.stdout:8/950: link dd/cdf dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/d47/c126 0 2024-03-11T19:56:49.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/880: sync 2024-03-11T19:56:49.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/992: sync 2024-03-11T19:56:49.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/951: mkdir dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/df2/d127 0 2024-03-11T19:56:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/952: mknod dd/d13/d18/d81/c128 0 2024-03-11T19:56:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/953: chown dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/df2 484453933 1 2024-03-11T19:56:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/954: fdatasync dd/d13/f6c 0 2024-03-11T19:56:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/955: truncate dd/de/d1b/f83 4797269 0 2024-03-11T19:56:49.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/881: read de/d7a/d104/fd6 [361257,57978] 0 2024-03-11T19:56:49.330 INFO:tasks.workunit.client.0.smithi107.stdout:0/882: link de/d11/d23/d32/d34/d39/f9f de/d11/d23/d32/d34/d39/d3a/d72/f10d 0 2024-03-11T19:56:49.339 INFO:tasks.workunit.client.0.smithi107.stdout:0/883: write de/d11/d23/d32/d34/d39/d3a/f5e [4333058,125321] 0 2024-03-11T19:56:49.356 INFO:tasks.workunit.client.0.smithi107.stdout:4/993: read d4/d7/d2f/fe8 [4097619,56431] 0 2024-03-11T19:56:49.389 INFO:tasks.workunit.client.0.smithi107.stdout:0/884: dread f3 [0,4194304] 0 2024-03-11T19:56:49.389 INFO:tasks.workunit.client.0.smithi107.stdout:0/885: fsync de/d11/d23/d32/d34/d39/d3a/f5e 0 2024-03-11T19:56:49.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/886: rmdir de/d11/d23/d32/d34/d39/d55 39 2024-03-11T19:56:49.392 INFO:tasks.workunit.client.0.smithi107.stdout:8/956: dwrite dd/f21 [0,4194304] 0 2024-03-11T19:56:49.392 INFO:tasks.workunit.client.0.smithi107.stdout:8/957: fsync dd/de/d1b/d4b/fb7 0 2024-03-11T19:56:49.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/887: link de/d11/d23/d32/f103 de/d11/d23/d24/d83/f10e 0 2024-03-11T19:56:49.429 INFO:tasks.workunit.client.0.smithi107.stdout:4/994: dwrite d4/d7/d2f/d3b/d101/d24/f103 [0,4194304] 0 2024-03-11T19:56:49.429 INFO:tasks.workunit.client.0.smithi107.stdout:4/995: stat d4/d7/d2f/d3b/d101/d24/f6a 0 2024-03-11T19:56:49.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/958: write dd/f21 [1154299,59290] 0 2024-03-11T19:56:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/888: dread de/d11/d74/da2/dd3/dd4/f42 [0,4194304] 0 2024-03-11T19:56:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/889: dread - de/d11/d74/da3/ffa zero size 2024-03-11T19:56:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/996: unlink d4/d7/db/lf 0 2024-03-11T19:56:49.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/997: stat d4/d7/d2f/d3b/d101/d19/d75/d8e/l9e 0 2024-03-11T19:56:49.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/998: chown d4/d7/d2f/d3b/d101/l41 84 1 2024-03-11T19:56:49.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/999: write d4/d7/d2f/d3b/d101/d24/d26/d29/f30 [959024,68501] 0 2024-03-11T19:56:49.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/890: creat de/d7a/f10f x:0 0 0 2024-03-11T19:56:49.435 INFO:tasks.workunit.client.0.smithi107.stdout:0/891: creat de/d11/d23/d24/da4/f110 x:0 0 0 2024-03-11T19:56:49.435 INFO:tasks.workunit.client.0.smithi107.stdout:0/892: write de/d11/d23/d32/d34/d39/f6e [118989,129830] 0 2024-03-11T19:56:49.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/959: rename dd/d13/d18/dd0/cd2 to dd/de/d1b/d20/d56/da4/dee/d120/d90/d96/c129 0 2024-03-11T19:56:49.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/960: creat dd/d13/d18/dd0/f12a x:0 0 0 2024-03-11T19:56:49.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/893: dwrite de/d11/f35 [0,4194304] 0 2024-03-11T19:56:49.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/894: rmdir de/d11/d23/d32/d34/d39/d55 39 2024-03-11T19:56:49.490 INFO:tasks.workunit.client.0.smithi107.stdout:0/895: link de/d58/fa7 de/f111 0 2024-03-11T19:56:49.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/961: rename dd/d13/f3d to dd/de/d1b/d20/d56/da4/dee/d120/d90/d124/f12b 0 2024-03-11T19:56:49.492 INFO:tasks.workunit.client.0.smithi107.stdout:0/896: creat de/d11/d23/d32/d34/d39/d55/d7c/f112 x:0 0 0 2024-03-11T19:56:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/962: mknod dd/de/d24/d115/d119/c12c 0 2024-03-11T19:56:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/963: readlink dd/dac/lbc 0 2024-03-11T19:56:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/964: stat dd/de/dfc/d10f/d7e/fae 0 2024-03-11T19:56:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/965: write dd/de/d1b/f1d [3358769,34043] 0 2024-03-11T19:56:49.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/897: stat de/d9e/dc8/fe0 0 2024-03-11T19:56:49.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/966: symlink dd/de/d1b/d20/l12d 0 2024-03-11T19:56:49.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/898: getdents de/d11/d23/d32/d34/d39/d55/d7c 0 2024-03-11T19:56:49.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/899: getdents de/d11/d74/da2/dd3/dd4/dbf 0 2024-03-11T19:56:49.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/967: dread dd/d13/d32/f89 [0,4194304] 0 2024-03-11T19:56:49.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/968: dread - dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/d47/fb0 zero size 2024-03-11T19:56:49.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/969: readlink dd/dac/lc6 0 2024-03-11T19:56:49.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/970: stat dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/d47/cfd 0 2024-03-11T19:56:49.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/971: chown dd/d13/f52 62 1 2024-03-11T19:56:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/900: dwrite de/f6a [0,4194304] 0 2024-03-11T19:56:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/901: write de/d11/d23/d24/f2d [2775085,31581] 0 2024-03-11T19:56:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/902: dread - de/d11/d23/d32/f103 zero size 2024-03-11T19:56:49.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/903: chown de/d11/d23/d24/d4f/d89/lfb 2272 1 2024-03-11T19:56:49.547 INFO:tasks.workunit.client.0.smithi107.stdout:0/904: creat de/d11/d23/d32/d34/d39/d55/d7c/f113 x:0 0 0 2024-03-11T19:56:49.547 INFO:tasks.workunit.client.0.smithi107.stdout:0/905: write de/d11/d23/d32/d34/f9b [280346,102950] 0 2024-03-11T19:56:49.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/906: unlink de/d11/d23/d24/d4f/d89/cc5 0 2024-03-11T19:56:49.571 INFO:tasks.workunit.client.0.smithi107.stdout:8/972: dread dd/de/d1b/f7a [4194304,4194304] 0 2024-03-11T19:56:49.624 INFO:tasks.workunit.client.0.smithi107.stdout:0/907: dwrite de/d11/d74/da2/fb1 [0,4194304] 0 2024-03-11T19:56:49.624 INFO:tasks.workunit.client.0.smithi107.stdout:0/908: readlink l7 0 2024-03-11T19:56:49.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/973: dwrite dd/de/dfc/d10f/f60 [0,4194304] 0 2024-03-11T19:56:49.637 INFO:tasks.workunit.client.0.smithi107.stdout:0/909: link de/d11/d23/d32/d34/d39/cad de/d11/d23/d24/d4f/d89/c114 0 2024-03-11T19:56:49.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/910: chown de/d11/d74/da3/cfc 1337096304 1 2024-03-11T19:56:49.638 INFO:tasks.workunit.client.0.smithi107.stdout:8/974: fsync dd/de/d1b/d20/d56/da4/dee/d120/fc2 0 2024-03-11T19:56:49.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/911: mkdir de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda/d115 0 2024-03-11T19:56:49.639 INFO:tasks.workunit.client.0.smithi107.stdout:0/912: stat de/d11/d23/d32/d34/d39/d3a/f50 0 2024-03-11T19:56:49.640 INFO:tasks.workunit.client.0.smithi107.stdout:8/975: creat dd/de/d1b/d20/d56/da4/dee/d120/de8/d117/f12e x:0 0 0 2024-03-11T19:56:49.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/976: creat dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/de1/f12f x:0 0 0 2024-03-11T19:56:49.643 INFO:tasks.workunit.client.0.smithi107.stdout:8/977: creat dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/d47/f130 x:0 0 0 2024-03-11T19:56:49.644 INFO:tasks.workunit.client.0.smithi107.stdout:8/978: mknod dd/d13/d18/dd0/c131 0 2024-03-11T19:56:49.645 INFO:tasks.workunit.client.0.smithi107.stdout:8/979: creat dd/de/dfc/d10f/d46/f132 x:0 0 0 2024-03-11T19:56:49.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/980: creat dd/de/d1b/d20/d56/da4/dee/d111/f133 x:0 0 0 2024-03-11T19:56:49.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/913: dwrite de/d11/d23/d32/d34/d39/d55/d7c/dd7/fff [0,4194304] 0 2024-03-11T19:56:49.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/981: dwrite dd/d13/d32/f89 [0,4194304] 0 2024-03-11T19:56:49.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/982: mkdir dd/de/dfc/d10f/d7e/d92/d134 0 2024-03-11T19:56:49.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/983: chown dd/c17 600501703 1 2024-03-11T19:56:49.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/984: chown dd/d16/f33 2 1 2024-03-11T19:56:49.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/985: read - dd/dac/ff8 zero size 2024-03-11T19:56:49.735 INFO:tasks.workunit.client.0.smithi107.stdout:0/914: dwrite de/d11/d23/d32/d34/d39/d55/d7c/f112 [0,4194304] 0 2024-03-11T19:56:49.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/915: mkdir de/d11/d23/d24/d4f/d116 0 2024-03-11T19:56:49.739 INFO:tasks.workunit.client.0.smithi107.stdout:0/916: creat de/d11/d23/d32/d34/d39/d3a/f117 x:0 0 0 2024-03-11T19:56:49.748 INFO:tasks.workunit.client.0.smithi107.stdout:0/917: dread de/d7a/f85 [0,4194304] 0 2024-03-11T19:56:49.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/918: dwrite de/d11/d23/d32/f103 [0,4194304] 0 2024-03-11T19:56:49.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/919: dread de/d58/fa7 [0,4194304] 0 2024-03-11T19:56:49.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/920: creat de/d11/d23/d32/d34/d39/d3a/f118 x:0 0 0 2024-03-11T19:56:49.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/921: chown de/d7a/d104/leb 1 1 2024-03-11T19:56:49.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/922: mkdir de/d11/d23/d32/d34/d39/d55/d119 0 2024-03-11T19:56:49.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/923: mknod de/d11/d23/d32/d34/d39/d55/d7c/def/d101/c11a 0 2024-03-11T19:56:49.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/924: chown de/d11/d23/d24/c90 970043149 1 2024-03-11T19:56:49.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/925: dread - de/d11/d23/d24/d83/fab zero size 2024-03-11T19:56:49.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/926: write de/d11/d23/d32/d34/d39/d55/f95 [453073,15912] 0 2024-03-11T19:56:49.814 INFO:tasks.workunit.client.0.smithi107.stdout:0/927: mknod de/d11/d23/c11b 0 2024-03-11T19:56:49.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/928: mknod de/d11/d23/d24/da4/dae/dcd/d109/c11c 0 2024-03-11T19:56:49.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/929: truncate de/d11/d23/d24/da4/dae/fb8 656416 0 2024-03-11T19:56:49.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/930: write de/f44 [2835828,45577] 0 2024-03-11T19:56:49.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/931: creat de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda/d115/f11d x:0 0 0 2024-03-11T19:56:49.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/932: write de/d7a/f85 [1142814,38515] 0 2024-03-11T19:56:49.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/933: readlink de/d11/d23/d24/da4/dae/le2 0 2024-03-11T19:56:49.830 INFO:tasks.workunit.client.0.smithi107.stdout:0/934: chown de/d11/c22 243883 1 2024-03-11T19:56:49.831 INFO:tasks.workunit.client.0.smithi107.stdout:0/935: chown de/d11/d74/da2/dd3/dd4/c49 262645 1 2024-03-11T19:56:49.888 INFO:tasks.workunit.client.0.smithi107.stdout:0/936: dwrite de/d11/f4c [0,4194304] 0 2024-03-11T19:56:49.888 INFO:tasks.workunit.client.0.smithi107.stdout:0/937: creat de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda/d115/f11e x:0 0 0 2024-03-11T19:56:49.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/938: truncate de/d11/d23/d32/d34/d39/f53 4364350 0 2024-03-11T19:56:49.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/939: symlink de/d11/d23/d32/d34/d39/d55/d7c/dd7/dda/l11f 0 2024-03-11T19:56:49.896 INFO:tasks.workunit.client.0.smithi107.stdout:8/986: sync 2024-03-11T19:56:49.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/940: truncate de/d11/d23/d32/d34/d39/f53 1336543 0 2024-03-11T19:56:49.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/987: mkdir dd/d13/d32/d51/d105/d135 0 2024-03-11T19:56:49.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/941: dwrite de/d11/d23/d32/d34/f9b [0,4194304] 0 2024-03-11T19:56:49.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/942: readlink de/d11/d23/d24/da4/dae/ldc 0 2024-03-11T19:56:49.951 INFO:tasks.workunit.client.0.smithi107.stdout:0/943: symlink de/d11/d74/l120 0 2024-03-11T19:56:49.952 INFO:tasks.workunit.client.0.smithi107.stdout:0/944: write de/f21 [514985,102154] 0 2024-03-11T19:56:49.953 INFO:tasks.workunit.client.0.smithi107.stdout:0/945: symlink de/d11/d23/d32/d34/d39/d55/d7c/l121 0 2024-03-11T19:56:49.953 INFO:tasks.workunit.client.0.smithi107.stdout:0/946: readlink de/d7a/lb6 0 2024-03-11T19:56:49.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/947: getdents de/d11/d23/d32/d34/d39 0 2024-03-11T19:56:49.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/948: chown de/d7a/f10f 17125 1 2024-03-11T19:56:49.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/949: creat de/d11/d23/d24/d8d/f122 x:0 0 0 2024-03-11T19:56:49.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/950: chown de/d11/d23/d24/da4/dae/dcd/d109 32867796 1 2024-03-11T19:56:49.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/951: truncate de/d11/d23/d32/f103 2383151 0 2024-03-11T19:56:49.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/952: creat de/f123 x:0 0 0 2024-03-11T19:56:49.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/953: chown de/d11/d23/d24/d83/dc4/l102 964659 1 2024-03-11T19:56:50.036 INFO:tasks.workunit.client.0.smithi107.stdout:0/954: dwrite de/d11/d74/da3/ffa [0,4194304] 0 2024-03-11T19:56:50.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/988: sync 2024-03-11T19:56:50.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/989: mknod dd/de/d1b/d20/d56/da4/dee/d120/d90/d121/d47/dbb/c136 0 2024-03-11T19:56:50.054 INFO:tasks.workunit.client.0.smithi107.stdout:8/990: symlink dd/d13/l137 0 2024-03-11T19:56:50.064 INFO:tasks.workunit.client.0.smithi107.stdout:0/955: dread de/d11/d23/d32/d34/d39/f53 [0,4194304] 0 2024-03-11T19:56:50.072 INFO:tasks.workunit.client.0.smithi107.stdout:8/991: dread dd/de/dfc/d10f/f42 [0,4194304] 0 2024-03-11T19:56:50.073 INFO:tasks.workunit.client.0.smithi107.stdout:0/956: read de/d11/d23/d32/d34/d39/d3a/d72/f7d [393738,57887] 0 2024-03-11T19:56:50.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/957: chown de/d11/d23/d32/d34/de6 26004167 1 2024-03-11T19:56:50.076 INFO:tasks.workunit.client.0.smithi107.stdout:0/958: read de/d11/d23/d32/d34/d39/d3a/d72/fc1 [39666,105128] 0 2024-03-11T19:56:50.077 INFO:tasks.workunit.client.0.smithi107.stdout:0/959: write de/d11/d23/d24/ff6 [613329,89213] 0 2024-03-11T19:56:50.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/992: link dd/de/d1b/d20/d56/da4/dee/d120/l38 dd/de/d24/d115/d119/l138 0 2024-03-11T19:56:50.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/993: write dd/de/d1b/d20/d56/da4/dee/d120/fc2 [274816,80443] 0 2024-03-11T19:56:50.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/960: write de/d11/d23/d24/da4/fb3 [2490805,23744] 0 2024-03-11T19:56:50.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/994: rename dd/d13/d18/d81/l8a to dd/de/d1b/d20/d56/da4/dee/d120/de8/l139 0 2024-03-11T19:56:50.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/995: rename dd/de/d1b/d20/d5e/dc8 to dd/de/d1b/d20/d56/da4/dee/d120/de8/d13a 0 2024-03-11T19:56:50.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/996: chown dd/d16 8 1 2024-03-11T19:56:50.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/997: truncate dd/d13/f108 272965 0 2024-03-11T19:56:50.151 INFO:tasks.workunit.client.0.smithi107.stdout:8/998: rename dd/d13/f6c to dd/d13/d32/d51/d105/f13b 0 2024-03-11T19:56:50.154 INFO:tasks.workunit.client.0.smithi107.stdout:8/999: mkdir dd/d13/d32/d51/d105/d13c 0 2024-03-11T19:56:50.195 INFO:tasks.workunit.client.0.smithi107.stdout:0/961: dwrite de/d11/d23/d24/d83/dc4/f107 [0,4194304] 0 2024-03-11T19:56:50.201 INFO:tasks.workunit.client.0.smithi107.stdout:0/962: mknod de/d11/d23/d24/da4/dae/dcd/c124 0 2024-03-11T19:56:50.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/963: mknod de/d11/d23/d32/d34/d39/d3a/c125 0 2024-03-11T19:56:50.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/964: dwrite de/d11/d23/d32/d34/d39/d3a/d72/fb2 [0,4194304] 0 2024-03-11T19:56:50.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/965: write de/d11/f2c [5277112,31115] 0 2024-03-11T19:56:50.329 INFO:tasks.workunit.client.0.smithi107.stdout:0/966: symlink de/d11/d23/d24/d83/dc4/dd2/l126 0 2024-03-11T19:56:50.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/967: write de/d11/d23/d32/d34/f82 [4086246,19544] 0 2024-03-11T19:56:50.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/968: dwrite de/d9e/dc8/fe0 [0,4194304] 0 2024-03-11T19:56:50.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/969: creat de/d11/d23/d24/d83/dc4/f127 x:0 0 0 2024-03-11T19:56:50.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/970: mknod de/d58/c128 0 2024-03-11T19:56:50.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/971: readlink de/d11/d23/d32/d34/d39/d3a/d72/lc2 0 2024-03-11T19:56:50.537 INFO:tasks.workunit.client.0.smithi107.stdout:0/972: dwrite de/d11/d23/d24/da4/fb3 [0,4194304] 0 2024-03-11T19:56:50.537 INFO:tasks.workunit.client.0.smithi107.stdout:0/973: truncate de/f123 505179 0 2024-03-11T19:56:50.537 INFO:tasks.workunit.client.0.smithi107.stdout:0/974: creat de/d11/d23/d24/da4/f129 x:0 0 0 2024-03-11T19:56:50.573 INFO:tasks.workunit.client.0.smithi107.stdout:0/975: dread de/d11/d23/d32/d34/d39/d3a/d72/f10d [0,4194304] 0 2024-03-11T19:56:50.573 INFO:tasks.workunit.client.0.smithi107.stdout:0/976: readlink de/d11/d23/d24/d4f/l6d 0 2024-03-11T19:56:50.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/977: creat de/d11/d23/f12a x:0 0 0 2024-03-11T19:56:50.671 INFO:tasks.workunit.client.0.smithi107.stdout:0/978: sync 2024-03-11T19:56:50.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/979: dwrite de/d11/d23/d32/d34/d39/d3a/f66 [0,4194304] 0 2024-03-11T19:56:50.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/980: dwrite de/d11/d74/fa5 [0,4194304] 0 2024-03-11T19:56:50.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/981: symlink de/d11/d74/da2/dd3/dd4/dbf/l12b 0 2024-03-11T19:56:50.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/982: dread - de/d9e/f100 zero size 2024-03-11T19:56:50.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/983: chown de/d11/ldb 9 1 2024-03-11T19:56:50.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/984: rename de/d11/d74/ca9 to de/d11/d23/d24/d8d/c12c 0 2024-03-11T19:56:50.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/985: fdatasync de/f123 0 2024-03-11T19:56:50.857 INFO:tasks.workunit.client.0.smithi107.stdout:0/986: readlink l2 0 2024-03-11T19:56:50.860 INFO:tasks.workunit.client.0.smithi107.stdout:0/987: symlink de/d11/d23/d32/d34/d39/d55/d7c/dd7/l12d 0 2024-03-11T19:56:50.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/988: dwrite de/d11/d23/d24/f69 [0,4194304] 0 2024-03-11T19:56:50.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/989: creat de/d9e/f12e x:0 0 0 2024-03-11T19:56:50.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/990: fdatasync de/d11/d23/d24/d8d/fe5 0 2024-03-11T19:56:50.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/991: write de/d11/d23/d32/d34/d39/d3a/d72/fb2 [1354995,14168] 0 2024-03-11T19:56:50.973 INFO:tasks.workunit.client.0.smithi107.stdout:0/992: creat de/d11/d23/f12f x:0 0 0 2024-03-11T19:56:50.973 INFO:tasks.workunit.client.0.smithi107.stdout:0/993: creat de/d11/d23/d24/da4/f130 x:0 0 0 2024-03-11T19:56:50.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/994: read de/f88 [106399,50301] 0 2024-03-11T19:56:51.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/995: dread de/d11/d23/d24/ff6 [0,4194304] 0 2024-03-11T19:56:51.002 INFO:tasks.workunit.client.0.smithi107.stdout:0/996: creat de/d11/d23/d24/d4f/d89/f131 x:0 0 0 2024-03-11T19:56:51.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/997: creat de/d11/d23/d24/d4f/f132 x:0 0 0 2024-03-11T19:56:51.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/998: dread - de/d11/d23/f12a zero size 2024-03-11T19:56:51.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/999: chown de/d7a/d104 3574 1 2024-03-11T19:56:51.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:50 smithi122 ceph-mon[36397]: pgmap v105: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 92 op/s 2024-03-11T19:56:51.012 INFO:tasks.workunit.client.0.smithi107.stderr:+ rm -rf -- ./tmp.vcE6wKMJIu 2024-03-11T19:56:51.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:50 smithi145 ceph-mon[36730]: pgmap v105: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 92 op/s 2024-03-11T19:56:51.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:50 smithi107 ceph-mon[31712]: pgmap v105: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 92 op/s 2024-03-11T19:56:53.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:52 smithi122 ceph-mon[36397]: pgmap v106: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 29 MiB/s wr, 232 op/s 2024-03-11T19:56:53.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:52 smithi145 ceph-mon[36730]: pgmap v106: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 29 MiB/s wr, 232 op/s 2024-03-11T19:56:53.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:52 smithi107 ceph-mon[31712]: pgmap v106: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 29 MiB/s wr, 232 op/s 2024-03-11T19:56:55.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:54 smithi122 ceph-mon[36397]: pgmap v107: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 248 op/s 2024-03-11T19:56:55.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:54 smithi145 ceph-mon[36730]: pgmap v107: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 248 op/s 2024-03-11T19:56:55.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:54 smithi107 ceph-mon[31712]: pgmap v107: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 248 op/s 2024-03-11T19:56:57.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:56 smithi122 ceph-mon[36397]: pgmap v108: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 220 op/s 2024-03-11T19:56:57.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:56 smithi145 ceph-mon[36730]: pgmap v108: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 220 op/s 2024-03-11T19:56:57.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:56 smithi107 ceph-mon[31712]: pgmap v108: 129 pgs: 129 active+clean; 2.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 220 op/s 2024-03-11T19:56:59.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:58 smithi122 ceph-mon[36397]: pgmap v109: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 277 op/s 2024-03-11T19:56:59.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:58 smithi145 ceph-mon[36730]: pgmap v109: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 277 op/s 2024-03-11T19:56:59.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:58 smithi107 ceph-mon[31712]: pgmap v109: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 277 op/s 2024-03-11T19:56:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 -- 172.21.15.107:0/1049766908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06cd50 msgr2=0x7fc70c06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 --2- 172.21.15.107:0/1049766908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06cd50 0x7fc70c06d130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009900 tx=0x7fc6fc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 -- 172.21.15.107:0/1049766908 shutdown_connections 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 --2- 172.21.15.107:0/1049766908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c11a820 0x7fc70c11cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 --2- 172.21.15.107:0/1049766908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c06d670 0x7fc70c11a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 --2- 172.21.15.107:0/1049766908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06cd50 0x7fc70c06d130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 -- 172.21.15.107:0/1049766908 >> 172.21.15.107:0/1049766908 conn(0x7fc70c06c3b0 msgr2=0x7fc70c10a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.896+0000 7fc710e00640 1 -- 172.21.15.107:0/1049766908 shutdown_connections 2024-03-11T19:56:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.897+0000 7fc710e00640 1 -- 172.21.15.107:0/1049766908 wait complete. 2024-03-11T19:56:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.897+0000 7fc710e00640 1 Processor -- start 2024-03-11T19:56:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 -- start start 2024-03-11T19:56:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 0x7fc70c1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c11a820 0x7fc70c1aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1ab390 con 0x7fc70c11a820 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1ab500 con 0x7fc70c06cd50 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc710e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1ab6c0 con 0x7fc70c06d670 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc709400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc709e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 0x7fc70c1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc709400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:36796/0 (socket says 172.21.15.107:36796) 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc70a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c11a820 0x7fc70c1aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.898+0000 7fc709400640 1 -- 172.21.15.107:0/4136385552 learned_addr learned my addr 172.21.15.107:0/4136385552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 -- 172.21.15.107:0/4136385552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 msgr2=0x7fc70c1a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 -- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c11a820 msgr2=0x7fc70c1aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c11a820 0x7fc70c1aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 -- 172.21.15.107:0/4136385552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc009560 con 0x7fc70c06cd50 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709400640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc709e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 0x7fc70c1a64d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02c170 tx=0x7fc6fc02cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.899+0000 7fc702a00640 1 -- 172.21.15.107:0/4136385552 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc004350 con 0x7fc70c06cd50 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.900+0000 7fc702a00640 1 -- 172.21.15.107:0/4136385552 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc0044b0 con 0x7fc70c06cd50 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.900+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c1ab940 con 0x7fc70c06cd50 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.900+0000 7fc702a00640 1 -- 172.21.15.107:0/4136385552 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc003c60 con 0x7fc70c06cd50 2024-03-11T19:56:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.900+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c1b3220 con 0x7fc70c06cd50 2024-03-11T19:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc702a00640 1 -- 172.21.15.107:0/4136385552 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc6fc004df0 con 0x7fc70c06cd50 2024-03-11T19:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc713b64740 1 Processor -- start 2024-03-11T19:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc713b64740 1 -- start start 2024-03-11T19:56:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc713b64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc713b64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d029190230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.903+0000 7fc713b64740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d02919b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc713b64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02914b620 con 0x55d02918fdb0 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc713b64740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0290fd8c0 con 0x55d029199330 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc713b64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0290fadf0 con 0x55d029190b20 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc709e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d029190230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:36806/0 (socket says 172.21.15.107:36806) 2024-03-11T19:56:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 -- 172.21.15.107:0/1453877101 learned_addr learned my addr 172.21.15.107:0/1453877101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc709400640 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d02919b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 -- 172.21.15.107:0/1453877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 msgr2=0x55d02919b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d02919b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 -- 172.21.15.107:0/1453877101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 msgr2=0x55d029190230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d029190230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc70a800640 1 -- 172.21.15.107:0/1453877101 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d029077b00 con 0x55d029190b20 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.904+0000 7fc709400640 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d02919b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc70a800640 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc704009c20 tx=0x7fc704011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a78563c3aeb4663 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc6f7400640 1 -- 172.21.15.107:0/1453877101 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704019630 con 0x55d029190b20 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc702a00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 0x7fc6f00a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc6f7400640 1 -- 172.21.15.107:0/1453877101 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc704009e90 con 0x55d029190b20 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc6f7400640 1 -- 172.21.15.107:0/1453877101 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400eaf0 con 0x55d029190b20 2024-03-11T19:56:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc702a00640 1 -- 172.21.15.107:0/4136385552 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc6fc0f0120 con 0x7fc70c06cd50 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.905+0000 7fc709e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 0x7fc6f00a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 -- 172.21.15.107:0/1453877101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 msgr2=0x55d02918f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc704009c20 tx=0x7fc704011810 comp rx=0 tx=0).stop 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 -- 172.21.15.107:0/1453877101 shutdown_connections 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d02919b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d029190230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 --2- 172.21.15.107:0/1453877101 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d02918f870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc709e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 0x7fc6f00a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55d029191280 tx=0x7fc6fc0f3b90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 -- 172.21.15.107:0/1453877101 >> 172.21.15.107:0/1453877101 conn(0x55d0291574b0 msgr2=0x55d02918aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 -- 172.21.15.107:0/1453877101 shutdown_connections 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.906+0000 7fc713b64740 1 -- 172.21.15.107:0/1453877101 wait complete. 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.907+0000 7fc713b64740 1 -- start start 2024-03-11T19:56:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.907+0000 7fc713b64740 1 Processor -- start 2024-03-11T19:56:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d0291963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 0x55d029196920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 0x55d029196e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0290f65d0 con 0x55d02918fdb0 2024-03-11T19:56:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02909bb50 con 0x55d029190b20 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02909b7f0 con 0x55d029199330 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc713b64740 10 client.0 _ll_register_callbacks cb 0x55d0290af550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc70a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d0291963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc709400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 0x55d029196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc70a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d0291963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36068/0 (socket says 172.21.15.107:36068) 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.908+0000 7fc70a800640 1 -- 172.21.15.107:0/2145349132 learned_addr learned my addr 172.21.15.107:0/2145349132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc70a800640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 0x55d029196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 -- 172.21.15.107:0/2145349132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 msgr2=0x55d029196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 0x55d029196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 -- 172.21.15.107:0/2145349132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 msgr2=0x55d0291963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d0291963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 -- 172.21.15.107:0/2145349132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040098d0 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc70a800640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 0x55d029196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc709400640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 0x55d029196920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55d02918bc30 tx=0x7fc6f800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b23fe20c373bca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc6ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.909+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8004d20 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc6f8017070 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0290c8cd0 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f8004e80 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc713b64740 10 client.25775 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d0290f4c50 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.910+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d0290c8cd0 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc6f8011740 con 0x55d029190b20 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 10 client.25775 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 20 client.25775 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d0291a93d0 0x55d0291ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 4 client.25775 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0290ecce0 con 0x55d0291a93d0 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.911+0000 7fc70a800640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d0291a93d0 0x55d0291ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc70a800640 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d0291a93d0 0x55d0291ab7f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a4b82e73dfd48e server_cookie=2c8af5837bcdf380 in_seq=0 out_seq=0 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc6ebe00640 10 client.25775 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc6ebe00640 1 -- 172.21.15.107:0/2145349132 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0290ecce0 con 0x55d0291a93d0 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc6ebe00640 10 client.25775 handle_command_reply: tid=0 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc713b64740 1 client.25775 shutdown 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 msgr2=0x55d029196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.912+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 0x55d029196920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55d02918bc30 tx=0x7fc6f800e720 comp rx=0 tx=0).stop 2024-03-11T19:56:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 shutdown_connections 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d0291a93d0 0x55d0291ab7f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029190b20 0x55d029196920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029199330 0x55d029196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 --2- 172.21.15.107:0/2145349132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d02918fdb0 0x55d0291963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc6ebe00640 0 client.25775 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:56:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 >> 172.21.15.107:0/2145349132 conn(0x55d0291574b0 msgr2=0x55d02918c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 shutdown_connections 2024-03-11T19:56:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 1 -- 172.21.15.107:0/2145349132 wait complete. 2024-03-11T19:56:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.913+0000 7fc713b64740 20 client.25775 trim_cache size 0 max 16384 2024-03-11T19:56:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 Processor -- start 2024-03-11T19:56:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 -- start start 2024-03-11T19:56:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d02918fdb0 0x55d029198b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 0x55d0291a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d028e3ee60 con 0x55d029199330 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0290ecce0 con 0x55d02918fdb0 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.934+0000 7fc713b64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0291912b0 con 0x55d029190b20 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc70a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d02918fdb0 0x55d029198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:36822/0 (socket says 172.21.15.107:36822) 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 -- 172.21.15.107:0/3804514325 learned_addr learned my addr 172.21.15.107:0/3804514325 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc70a800640 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 0x55d0291a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 -- 172.21.15.107:0/3804514325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d02918fdb0 msgr2=0x55d029198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d02918fdb0 0x55d029198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 -- 172.21.15.107:0/3804514325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 msgr2=0x55d0291a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 0x55d0291a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 -- 172.21.15.107:0/3804514325 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d029091530 con 0x55d029190b20 2024-03-11T19:56:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc70a800640 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 0x55d0291a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:56:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.935+0000 7fc709400640 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8007960 tx=0x7fc6f800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74f1a61f7608f8c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.936+0000 7fc6f6a00640 1 -- 172.21.15.107:0/3804514325 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80049e0 con 0x55d029190b20 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.936+0000 7fc6f6a00640 1 -- 172.21.15.107:0/3804514325 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f8015070 con 0x55d029190b20 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.936+0000 7fc6f6a00640 1 -- 172.21.15.107:0/3804514325 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8028800 con 0x55d029190b20 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- 172.21.15.107:0/3804514325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 msgr2=0x55d0291a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8007960 tx=0x7fc6f800eca0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- 172.21.15.107:0/3804514325 shutdown_connections 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029199330 0x55d0291a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d029190b20 0x55d0291a0150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 --2- 172.21.15.107:0/3804514325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d02918fdb0 0x55d029198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- 172.21.15.107:0/3804514325 >> 172.21.15.107:0/3804514325 conn(0x55d0291574b0 msgr2=0x55d0291b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- 172.21.15.107:0/3804514325 shutdown_connections 2024-03-11T19:56:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- 172.21.15.107:0/3804514325 wait complete. 2024-03-11T19:56:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.937+0000 7fc713b64740 1 -- start start 2024-03-11T19:56:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 Processor -- start 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029190b20 0x55d02919c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d0291fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0291fee00 con 0x55d029190b20 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0291fef40 con 0x55d029199330 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0291ff080 con 0x55d02918fdb0 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc713b64740 10 client.0 _ll_register_callbacks cb 0x7fc6f8005c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc70a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc70a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:36826/0 (socket says 172.21.15.107:36826) 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc70a800640 1 -- 172.21.15.107:0/103094474 learned_addr learned my addr 172.21.15.107:0/103094474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc70a800640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d0291fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.938+0000 7fc709400640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029190b20 0x55d02919c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 -- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 msgr2=0x55d0291fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d0291fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 -- 172.21.15.107:0/103094474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029190b20 msgr2=0x55d02919c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029190b20 0x55d02919c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 -- 172.21.15.107:0/103094474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0290f4c50 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc70a800640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e660 tx=0x7fc70401b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b6d7b8f93e5b53 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc6ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704005610 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc70400e9d0 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02911bd30 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70400eb90 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.939+0000 7fc713b64740 10 client.25780 fetch_fsmap learned FSMap version 28 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d0290b72c0 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d02911bd30 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc704039020 con 0x55d02918fdb0 2024-03-11T19:56:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 10 client.25780 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 20 client.25780 populate_metadata read hostname 'smithi107' 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d029200710 0x55d029202b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 4 client.25780 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d028df6600 con 0x55d029200710 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.940+0000 7fc709400640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d029200710 0x55d029202b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.941+0000 7fc709400640 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d029200710 0x55d029202b30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ce9a76f4c6a003 server_cookie=3c2a0443e454c8ab in_seq=0 out_seq=0 2024-03-11T19:56:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:56:59.941+0000 7fc6ebe00640 10 client.25780 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.099+0000 7fc6ebe00640 1 -- 172.21.15.107:0/103094474 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d028df6600 con 0x55d029200710 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.099+0000 7fc6ebe00640 10 client.25780 handle_command_reply: tid=0 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.099+0000 7fc713b64740 1 client.25780 shutdown 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.099+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 msgr2=0x55d02919c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.099+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc70400e660 tx=0x7fc70401b040 comp rx=0 tx=0).stop 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 shutdown_connections 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55d029200710 0x55d029202b30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d029199330 0x55d0291fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d029190b20 0x55d02919c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 --2- 172.21.15.107:0/103094474 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55d02918fdb0 0x55d02919c320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc6ebe00640 0 client.25780 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 >> 172.21.15.107:0/103094474 conn(0x55d0291574b0 msgr2=0x55d02918a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 shutdown_connections 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 1 -- 172.21.15.107:0/103094474 wait complete. 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.100+0000 7fc713b64740 20 client.25780 trim_cache size 0 max 16384 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:57:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 msgr2=0x7fc6f00a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 0x7fc6f00a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55d029191280 tx=0x7fc6fc0f3b90 comp rx=0 tx=0).stop 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 msgr2=0x7fc70c1a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 0x7fc70c1a64d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc02c170 tx=0x7fc6fc02cd80 comp rx=0 tx=0).stop 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 shutdown_connections 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc6f00a1cc0 0x7fc6f00a4180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c11a820 0x7fc70c1aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc70c06d670 0x7fc70c1a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 --2- 172.21.15.107:0/4136385552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c06cd50 0x7fc70c1a64d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.101+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 >> 172.21.15.107:0/4136385552 conn(0x7fc70c06c3b0 msgr2=0x7fc70c11c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.103+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 shutdown_connections 2024-03-11T19:57:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:00.103+0000 7fc710e00640 1 -- 172.21.15.107:0/4136385552 wait complete. 2024-03-11T19:57:00.259 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:56:59 smithi122 ceph-mon[36397]: pgmap v110: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 233 op/s 2024-03-11T19:57:00.283 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:56:59 smithi145 ceph-mon[36730]: pgmap v110: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 233 op/s 2024-03-11T19:57:00.290 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:56:59 smithi107 ceph-mon[31712]: pgmap v110: 129 pgs: 129 active+clean; 2.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 233 op/s 2024-03-11T19:57:03.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:02 smithi122 ceph-mon[36397]: pgmap v111: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 351 op/s 2024-03-11T19:57:03.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:02 smithi145 ceph-mon[36730]: pgmap v111: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 351 op/s 2024-03-11T19:57:03.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:02 smithi107 ceph-mon[31712]: pgmap v111: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 351 op/s 2024-03-11T19:57:05.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:04 smithi122 ceph-mon[36397]: pgmap v112: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-03-11T19:57:05.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:04 smithi145 ceph-mon[36730]: pgmap v112: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-03-11T19:57:05.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:04 smithi107 ceph-mon[31712]: pgmap v112: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-03-11T19:57:07.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:06 smithi122 ceph-mon[36397]: pgmap v113: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 719 KiB/s wr, 229 op/s 2024-03-11T19:57:07.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:06 smithi145 ceph-mon[36730]: pgmap v113: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 719 KiB/s wr, 229 op/s 2024-03-11T19:57:07.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:06 smithi107 ceph-mon[31712]: pgmap v113: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 719 KiB/s wr, 229 op/s 2024-03-11T19:57:08.835 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:08 smithi145 ceph-mon[36730]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-03-11T19:57:09.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:08 smithi122 ceph-mon[36397]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-03-11T19:57:09.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:08 smithi107 ceph-mon[31712]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-03-11T19:57:11.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:10 smithi122 ceph-mon[36397]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 867 KiB/s wr, 249 op/s 2024-03-11T19:57:11.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:10 smithi145 ceph-mon[36730]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 867 KiB/s wr, 249 op/s 2024-03-11T19:57:11.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:10 smithi107 ceph-mon[31712]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 867 KiB/s wr, 249 op/s 2024-03-11T19:57:13.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:12 smithi122 ceph-mon[36397]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-03-11T19:57:13.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:12 smithi145 ceph-mon[36730]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-03-11T19:57:13.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:12 smithi107 ceph-mon[31712]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 -- 172.21.15.107:0/886085369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d70101b80 msgr2=0x7f4d70105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 --2- 172.21.15.107:0/886085369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d70101b80 0x7f4d70105bc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009930 tx=0x7f4d5c02bc20 comp rx=0 tx=0).stop 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 -- 172.21.15.107:0/886085369 shutdown_connections 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 --2- 172.21.15.107:0/886085369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d70101b80 0x7f4d70105bc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 --2- 172.21.15.107:0/886085369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d701062a0 0x7f4d7010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 --2- 172.21.15.107:0/886085369 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701011d0 0x7f4d701015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 -- 172.21.15.107:0/886085369 >> 172.21.15.107:0/886085369 conn(0x7f4d700fca90 msgr2=0x7f4d700feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 -- 172.21.15.107:0/886085369 shutdown_connections 2024-03-11T19:57:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.246+0000 7f4d74a00640 1 -- 172.21.15.107:0/886085369 wait complete. 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.247+0000 7f4d74a00640 1 Processor -- start 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 -- start start 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 0x7f4d7019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 0x7f4d70199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019a1f0 con 0x7f4d70199720 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019a360 con 0x7f4d701011d0 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d74a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7019a520 con 0x7f4d701062a0 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36138/0 (socket says 172.21.15.107:36138) 2024-03-11T19:57:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.248+0000 7f4d6f400640 1 -- 172.21.15.107:0/3198969780 learned_addr learned my addr 172.21.15.107:0/3198969780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 0x7f4d70199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 0x7f4d7019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 -- 172.21.15.107:0/3198969780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 msgr2=0x7f4d7019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 0x7f4d7019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 -- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 msgr2=0x7f4d70199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 0x7f4d70199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 -- 172.21.15.107:0/3198969780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c009560 con 0x7f4d701011d0 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 0x7f4d70199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 0x7f4d7019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d6f400640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e9f0 tx=0x7f4d6000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.249+0000 7f4d67e00640 1 -- 172.21.15.107:0/3198969780 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60009720 con 0x7f4d701011d0 2024-03-11T19:57:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.250+0000 7f4d67e00640 1 -- 172.21.15.107:0/3198969780 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d6000cd80 con 0x7f4d701011d0 2024-03-11T19:57:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.250+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d701a6420 con 0x7f4d701011d0 2024-03-11T19:57:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.250+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d701a6900 con 0x7f4d701011d0 2024-03-11T19:57:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.250+0000 7f4d67e00640 1 -- 172.21.15.107:0/3198969780 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60010430 con 0x7f4d701011d0 2024-03-11T19:57:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.253+0000 7f4d67e00640 1 -- 172.21.15.107:0/3198969780 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4d600046c0 con 0x7f4d701011d0 2024-03-11T19:57:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.253+0000 7f4d78d9e740 1 Processor -- start 2024-03-11T19:57:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.253+0000 7f4d78d9e740 1 -- start start 2024-03-11T19:57:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbce5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbca1620 con 0x55c3bbce6d60 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbc538c0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbc50df0 con 0x55c3bbcef570 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbce5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36150/0 (socket says 172.21.15.107:36150) 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33380/0 (socket says 172.21.15.107:33380) 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6ea00640 1 -- 172.21.15.107:0/869314040 learned_addr learned my addr 172.21.15.107:0/869314040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbce5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49342/0 (socket says 172.21.15.107:49342) 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d67e00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 0x7f4d540a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6ea00640 1 -- 172.21.15.107:0/869314040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 msgr2=0x55c3bbce6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.254+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6ea00640 1 -- 172.21.15.107:0/869314040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 msgr2=0x55c3bbce5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbce5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6ea00640 1 -- 172.21.15.107:0/869314040 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3bbbcdb00 con 0x55c3bbcef570 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6f400640 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d67e00640 1 -- 172.21.15.107:0/3198969780 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4d60014070 con 0x7f4d701011d0 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c006010 tx=0x7f4d5c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d7e98310e7a1530 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 0x7f4d540a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d5be00640 1 -- 172.21.15.107:0/869314040 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c004a80 con 0x55c3bbcef570 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d5be00640 1 -- 172.21.15.107:0/869314040 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c004030 con 0x55c3bbcef570 2024-03-11T19:57:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.255+0000 7f4d5be00640 1 -- 172.21.15.107:0/869314040 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c005370 con 0x55c3bbcef570 2024-03-11T19:57:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 -- 172.21.15.107:0/869314040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 msgr2=0x55c3bbcf1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 0x7f4d540a40d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005fd0 tx=0x7f4d68004a40 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c006010 tx=0x7f4d5c005e80 comp rx=0 tx=0).stop 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 -- 172.21.15.107:0/869314040 shutdown_connections 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce5ff0 0x55c3bbce6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbcf1960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/869314040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbce5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.256+0000 7f4d78d9e740 1 -- 172.21.15.107:0/869314040 >> 172.21.15.107:0/869314040 conn(0x55c3bbcad4b0 msgr2=0x55c3bbce0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.257+0000 7f4d78d9e740 1 -- 172.21.15.107:0/869314040 shutdown_connections 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.257+0000 7f4d78d9e740 1 -- 172.21.15.107:0/869314040 wait complete. 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.257+0000 7f4d78d9e740 1 -- start start 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 Processor -- start 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbceca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbcef570 0x55c3bbcecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbc4c5d0 con 0x55c3bbce6d60 2024-03-11T19:57:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbbf1b50 con 0x55c3bbcef570 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbbf17f0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d78d9e740 10 client.0 _ll_register_callbacks cb 0x55c3bbc05550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d6f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33396/0 (socket says 172.21.15.107:33396) 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d6f400640 1 -- 172.21.15.107:0/1608967919 learned_addr learned my addr 172.21.15.107:0/1608967919 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbcef570 0x55c3bbcecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.258+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbceca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 -- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbcef570 msgr2=0x55c3bbcecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbcef570 0x55c3bbcecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 -- 172.21.15.107:0/1608967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 msgr2=0x55c3bbceca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbceca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 -- 172.21.15.107:0/1608967919 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7010e680 con 0x55c3bbce5ff0 2024-03-11T19:57:13.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d600cb370 tx=0x7f4d60096490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f7b3f224b2eb10 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6009a5c0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4d6009aef0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3bbc1ecd0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d6009a330 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d78d9e740 10 client.25789 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.259+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3bbc4ac50 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3bbc1ecd0 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4d6001f060 con 0x55c3bbce5ff0 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 10 client.25789 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 10 client.25789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 20 client.25789 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbcff610 0x55c3bbd01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 4 client.25789 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:57:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3bbc42ce0 con 0x55c3bbcff610 2024-03-11T19:57:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.260+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbcff610 0x55c3bbd01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbcff610 0x55c3bbd01a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce06024ea0f7b70 server_cookie=5544d2feddded8c6 in_seq=0 out_seq=0 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d58c00640 10 client.25789 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d58c00640 1 -- 172.21.15.107:0/1608967919 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3bbc42ce0 con 0x55c3bbcff610 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d58c00640 10 client.25789 handle_command_reply: tid=0 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d78d9e740 1 client.25789 shutdown 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 msgr2=0x55c3bbcec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.261+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d600cb370 tx=0x7f4d60096490 comp rx=0 tx=0).stop 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 shutdown_connections 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbcff610 0x55c3bbd01a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbcef570 0x55c3bbcecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce6d60 0x55c3bbceca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d58c00640 0 client.25789 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1608967919 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcec540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 >> 172.21.15.107:0/1608967919 conn(0x55c3bbcad4b0 msgr2=0x55c3bbce2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 shutdown_connections 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1608967919 wait complete. 2024-03-11T19:57:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.262+0000 7f4d78d9e740 20 client.25789 trim_cache size 0 max 16384 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 Processor -- start 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 -- start start 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce5ff0 0x55c3bbceec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbce97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bb994e60 con 0x55c3bbce5ff0 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbcf2180 con 0x55c3bbce6d60 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbcfddb0 con 0x55c3bbcef570 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbce97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36164/0 (socket says 172.21.15.107:36164) 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbce97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33406/0 (socket says 172.21.15.107:33406) 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1177314682 learned_addr learned my addr 172.21.15.107:0/1177314682 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.278+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce5ff0 0x55c3bbceec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1177314682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 msgr2=0x55c3bbce97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbce97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1177314682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce5ff0 msgr2=0x55c3bbceec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce5ff0 0x55c3bbceec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1177314682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3bbbe7530 con 0x55c3bbce6d60 2024-03-11T19:57:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c3bbce6a90 tx=0x7f4d5c02e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d763d14356f37570 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d5b400640 1 -- 172.21.15.107:0/1177314682 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c02c2a0 con 0x55c3bbce6d60 2024-03-11T19:57:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d5b400640 1 -- 172.21.15.107:0/1177314682 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c04eb70 con 0x55c3bbce6d60 2024-03-11T19:57:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.279+0000 7f4d5b400640 1 -- 172.21.15.107:0/1177314682 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c04e5d0 con 0x55c3bbce6d60 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1177314682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 msgr2=0x55c3bbcecde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c3bbce6a90 tx=0x7f4d5c02e040 comp rx=0 tx=0).stop 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1177314682 shutdown_connections 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcecde0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbcef570 0x55c3bbce97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1177314682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbce5ff0 0x55c3bbceec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1177314682 >> 172.21.15.107:0/1177314682 conn(0x55c3bbcad4b0 msgr2=0x55c3bbceae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1177314682 shutdown_connections 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.280+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1177314682 wait complete. 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 -- start start 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 Processor -- start 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcf4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbcef570 0x55c3bbcf57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbcf6c90 con 0x55c3bbcef570 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbd5e350 con 0x55c3bbce6d60 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3bbd5e490 con 0x55c3bbce5ff0 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.281+0000 7f4d78d9e740 10 client.0 _ll_register_callbacks cb 0x7f4d600d1580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:36170/0 (socket says 172.21.15.107:36170) 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1802869386 learned_addr learned my addr 172.21.15.107:0/1802869386 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6fe00640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbcef570 0x55c3bbcf57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcf4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1802869386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 msgr2=0x55c3bbcf4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcf4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1802869386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbcef570 msgr2=0x55c3bbcf57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbcef570 0x55c3bbcf57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 -- 172.21.15.107:0/1802869386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3bbbcdb00 con 0x55c3bbce6d60 2024-03-11T19:57:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d6ea00640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c02c160 tx=0x7f4d5c0042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da782c6aa57b0884 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.282+0000 7f4d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:57:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c02c9d0 con 0x55c3bbce6d60 2024-03-11T19:57:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3bbc71d30 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4d5c04c070 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d5c02cb30 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 10 client.25799 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3bbc0d2c0 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3bbc71d30 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4d5c055070 con 0x55c3bbce6d60 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 10 client.25799 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.283+0000 7f4d78d9e740 20 client.25799 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbd5f850 0x55c3bbd61c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d78d9e740 4 client.25799 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3bb94c600 con 0x55c3bbd5f850 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbd5f850 0x55c3bbd61c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d6f400640 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbd5f850 0x55c3bbd61c40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f4871fc5d68080 server_cookie=bb40819f22fce3df in_seq=0 out_seq=0 2024-03-11T19:57:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.284+0000 7f4d58c00640 10 client.25799 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d58c00640 1 -- 172.21.15.107:0/1802869386 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3bb94c600 con 0x55c3bbd5f850 2024-03-11T19:57:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d58c00640 10 client.25799 handle_command_reply: tid=0 2024-03-11T19:57:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 client.25799 shutdown 2024-03-11T19:57:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 msgr2=0x55c3bbcf5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c02c160 tx=0x7f4d5c0042a0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 shutdown_connections 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55c3bbd5f850 0x55c3bbd61c40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3bbce6d60 0x55c3bbcf5280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c3bbcef570 0x55c3bbcf57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 --2- 172.21.15.107:0/1802869386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c3bbce5ff0 0x55c3bbcf4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d58c00640 0 client.25799 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 >> 172.21.15.107:0/1802869386 conn(0x55c3bbcad4b0 msgr2=0x55c3bbcdfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 shutdown_connections 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.506+0000 7f4d78d9e740 1 -- 172.21.15.107:0/1802869386 wait complete. 2024-03-11T19:57:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.507+0000 7f4d78d9e740 20 client.25799 trim_cache size 0 max 16384 2024-03-11T19:57:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:57:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:57:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:57:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 msgr2=0x7f4d540a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 0x7f4d540a40d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005fd0 tx=0x7f4d68004a40 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 msgr2=0x7f4d7019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e9f0 tx=0x7f4d6000edf0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 shutdown_connections 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4d540a1c10 0x7f4d540a40d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d70199720 0x7f4d70199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4d701062a0 0x7f4d7019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 --2- 172.21.15.107:0/3198969780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d701011d0 0x7f4d7019f5c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 >> 172.21.15.107:0/3198969780 conn(0x7f4d700fca90 msgr2=0x7f4d700fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 shutdown_connections 2024-03-11T19:57:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:13.508+0000 7f4d74a00640 1 -- 172.21.15.107:0/3198969780 wait complete. 2024-03-11T19:57:15.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:14 smithi122 ceph-mon[36397]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 846 KiB/s wr, 198 op/s 2024-03-11T19:57:15.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:14 smithi145 ceph-mon[36730]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 846 KiB/s wr, 198 op/s 2024-03-11T19:57:15.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:14 smithi107 ceph-mon[31712]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 846 KiB/s wr, 198 op/s 2024-03-11T19:57:15.669 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T19:57:15.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T19:57:17.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:16 smithi122 ceph-mon[36397]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 708 KiB/s wr, 145 op/s 2024-03-11T19:57:17.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:16 smithi145 ceph-mon[36730]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 708 KiB/s wr, 145 op/s 2024-03-11T19:57:17.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:16 smithi107 ceph-mon[31712]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 708 KiB/s wr, 145 op/s 2024-03-11T19:57:19.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:18 smithi122 ceph-mon[36397]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 369 op/s 2024-03-11T19:57:19.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:18 smithi145 ceph-mon[36730]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 369 op/s 2024-03-11T19:57:19.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:18 smithi107 ceph-mon[31712]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 369 op/s 2024-03-11T19:57:21.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:20 smithi122 ceph-mon[36397]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 809 KiB/s wr, 292 op/s 2024-03-11T19:57:21.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:20 smithi145 ceph-mon[36730]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 809 KiB/s wr, 292 op/s 2024-03-11T19:57:21.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:20 smithi107 ceph-mon[31712]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 809 KiB/s wr, 292 op/s 2024-03-11T19:57:23.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:22 smithi122 ceph-mon[36397]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-03-11T19:57:23.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:22 smithi145 ceph-mon[36730]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-03-11T19:57:23.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:22 smithi107 ceph-mon[31712]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 313 op/s 2024-03-11T19:57:25.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:24 smithi122 ceph-mon[36397]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-03-11T19:57:25.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:24 smithi145 ceph-mon[36730]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-03-11T19:57:25.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:24 smithi107 ceph-mon[31712]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.653+0000 7f2d83200640 1 -- 172.21.15.107:0/1996568161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d841011f0 msgr2=0x7f2d841015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.653+0000 7f2d83200640 1 --2- 172.21.15.107:0/1996568161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d841011f0 0x7f2d841015d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009890 tx=0x7f2d7402bb70 comp rx=0 tx=0).stop 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 -- 172.21.15.107:0/1996568161 shutdown_connections 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 --2- 172.21.15.107:0/1996568161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 0x7f2d8410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 --2- 172.21.15.107:0/1996568161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d84101ba0 0x7f2d84105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 --2- 172.21.15.107:0/1996568161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d841011f0 0x7f2d841015d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 -- 172.21.15.107:0/1996568161 >> 172.21.15.107:0/1996568161 conn(0x7f2d840fca90 msgr2=0x7f2d840feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 -- 172.21.15.107:0/1996568161 shutdown_connections 2024-03-11T19:57:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.654+0000 7f2d83200640 1 -- 172.21.15.107:0/1996568161 wait complete. 2024-03-11T19:57:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.655+0000 7f2d83200640 1 Processor -- start 2024-03-11T19:57:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.655+0000 7f2d83200640 1 -- start start 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 0x7f2d84197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 0x7f2d84197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 0x7f2d8419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419c9b0 con 0x7f2d8419bfc0 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419cb20 con 0x7f2d841062c0 2024-03-11T19:57:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d83200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8419cc90 con 0x7f2d841011f0 2024-03-11T19:57:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 0x7f2d8419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 0x7f2d84197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 0x7f2d8419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45196/0 (socket says 172.21.15.107:45196) 2024-03-11T19:57:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d82800640 1 -- 172.21.15.107:0/140744644 learned_addr learned my addr 172.21.15.107:0/140744644 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.656+0000 7f2d81400640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 0x7f2d84197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 -- 172.21.15.107:0/140744644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 msgr2=0x7f2d84197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 0x7f2d84197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 -- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 msgr2=0x7f2d8419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 0x7f2d8419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 -- 172.21.15.107:0/140744644 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009560 con 0x7f2d841011f0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d81e00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 0x7f2d84197700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d74037040 tx=0x7f2d740045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d7aa00640 1 -- 172.21.15.107:0/140744644 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74034990 con 0x7f2d841011f0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d7aa00640 1 -- 172.21.15.107:0/140744644 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7402c9f0 con 0x7f2d841011f0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d7aa00640 1 -- 172.21.15.107:0/140744644 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74032e60 con 0x7f2d841011f0 2024-03-11T19:57:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d74009490 con 0x7f2d841011f0 2024-03-11T19:57:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.657+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841a4450 con 0x7f2d841011f0 2024-03-11T19:57:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.662+0000 7f2d7aa00640 1 -- 172.21.15.107:0/140744644 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2d7403b040 con 0x7f2d841011f0 2024-03-11T19:57:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.662+0000 7f2d8be60740 1 Processor -- start 2024-03-11T19:57:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.662+0000 7f2d8be60740 1 -- start start 2024-03-11T19:57:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983dc980 0x562a983ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9838a620 con 0x562a983ceff0 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9833c8c0 con 0x562a983dc980 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d8be60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a98339df0 con 0x562a983cfd60 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d81e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:48206/0 (socket says 172.21.15.107:48206) 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 -- 172.21.15.107:0/2635963985 learned_addr learned my addr 172.21.15.107:0/2635963985 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d81400640 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983dc980 0x562a983ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 -- 172.21.15.107:0/2635963985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983dc980 msgr2=0x562a983ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983dc980 0x562a983ded70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 -- 172.21.15.107:0/2635963985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 msgr2=0x562a983cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.663+0000 7f2d82800640 1 -- 172.21.15.107:0/2635963985 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a982b6b00 con 0x562a983cfd60 2024-03-11T19:57:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d7aa00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 0x7f2d680a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d7aa00640 1 -- 172.21.15.107:0/140744644 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2d740ef4b0 con 0x7f2d841011f0 2024-03-11T19:57:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d81e00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 0x7f2d680a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d82800640 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c011470 tx=0x7f2d7c011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38d7afbecd79846 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d6f400640 1 -- 172.21.15.107:0/2635963985 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004b20 con 0x562a983cfd60 2024-03-11T19:57:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d6f400640 1 -- 172.21.15.107:0/2635963985 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c004c80 con 0x562a983cfd60 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.664+0000 7f2d6f400640 1 -- 172.21.15.107:0/2635963985 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005370 con 0x562a983cfd60 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 -- 172.21.15.107:0/2635963985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 msgr2=0x562a983ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d81e00640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 0x7f2d680a4050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ecaa0 tx=0x7f2d740f50e0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c011470 tx=0x7f2d7c011930 comp rx=0 tx=0).stop 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 -- 172.21.15.107:0/2635963985 shutdown_connections 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983dc980 0x562a983ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 --2- 172.21.15.107:0/2635963985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983cfd60 0x562a983ceab0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 -- 172.21.15.107:0/2635963985 >> 172.21.15.107:0/2635963985 conn(0x562a983964b0 msgr2=0x562a983c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.665+0000 7f2d8be60740 1 -- 172.21.15.107:0/2635963985 shutdown_connections 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.666+0000 7f2d8be60740 1 -- 172.21.15.107:0/2635963985 wait complete. 2024-03-11T19:57:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.666+0000 7f2d8be60740 1 -- start start 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.666+0000 7f2d8be60740 1 Processor -- start 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 0x562a983d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a983355d0 con 0x562a983cfd60 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a982dab50 con 0x562a983ceff0 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a982da7f0 con 0x562a983dc980 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d8be60740 10 client.0 _ll_register_callbacks cb 0x562a982ee550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45212/0 (socket says 172.21.15.107:45212) 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 -- 172.21.15.107:0/1340836202 learned_addr learned my addr 172.21.15.107:0/1340836202 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d82800640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 0x562a983d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d82800640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 -- 172.21.15.107:0/1340836202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 msgr2=0x562a983da360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983da360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 -- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 msgr2=0x562a983d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 0x562a983d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d81400640 1 -- 172.21.15.107:0/1340836202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c009cd0 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.667+0000 7f2d82800640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 0x562a983d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d81400640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2d70011790 tx=0x7f2d70011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e570ea1c9a73ff39 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70011ea0 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2d7000fb30 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a98307cd0 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d70016070 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d8be60740 10 client.15681 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.668+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562a98333c50 con 0x562a983cfd60 2024-03-11T19:57:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562a98307cd0 con 0x562a983cfd60 2024-03-11T19:57:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2d70003c30 con 0x562a983cfd60 2024-03-11T19:57:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 10 client.15681 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 20 client.15681 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a983ec9d0 0x562a983eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 4 client.15681 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a9832bce0 con 0x562a983ec9d0 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.669+0000 7f2d82800640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a983ec9d0 0x562a983eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.670+0000 7f2d82800640 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a983ec9d0 0x562a983eedf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82afd2013f4b88e2 server_cookie=5a2551e5a7b3c18d in_seq=0 out_seq=0 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.670+0000 7f2d63e00640 10 client.15681 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.670+0000 7f2d63e00640 1 -- 172.21.15.107:0/1340836202 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a9832bce0 con 0x562a983ec9d0 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d63e00640 10 client.15681 handle_command_reply: tid=0 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 client.15681 shutdown 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 msgr2=0x562a983d9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2d70011790 tx=0x7f2d70011c50 comp rx=0 tx=0).stop 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 shutdown_connections 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a983ec9d0 0x562a983eedf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983da360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983cfd60 0x562a983d9e20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1340836202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983ceff0 0x562a983d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d63e00640 0 client.15681 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 >> 172.21.15.107:0/1340836202 conn(0x562a983964b0 msgr2=0x562a983cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 shutdown_connections 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 1 -- 172.21.15.107:0/1340836202 wait complete. 2024-03-11T19:57:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.671+0000 7f2d8be60740 20 client.15681 trim_cache size 0 max 16384 2024-03-11T19:57:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 Processor -- start 2024-03-11T19:57:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 -- start start 2024-03-11T19:57:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9807de60 con 0x562a983ceff0 2024-03-11T19:57:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9832bce0 con 0x562a983cfd60 2024-03-11T19:57:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.686+0000 7f2d8be60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a98307cd0 con 0x562a983dc980 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983dc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45222/0 (socket says 172.21.15.107:45222) 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d82800640 1 -- 172.21.15.107:0/1210028821 learned_addr learned my addr 172.21.15.107:0/1210028821 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d82800640 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 -- 172.21.15.107:0/1210028821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 msgr2=0x562a983d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983d6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 -- 172.21.15.107:0/1210028821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 msgr2=0x562a983dc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983dc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 -- 172.21.15.107:0/1210028821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a982d0530 con 0x562a983cfd60 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d82800640 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.687+0000 7f2d81400640 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983da180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562a983daeb0 tx=0x7f2d7000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ea0db3b0aafe74e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.688+0000 7f2d6ea00640 1 -- 172.21.15.107:0/1210028821 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70011ea0 con 0x562a983cfd60 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.688+0000 7f2d6ea00640 1 -- 172.21.15.107:0/1210028821 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7001ae40 con 0x562a983cfd60 2024-03-11T19:57:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.688+0000 7f2d6ea00640 1 -- 172.21.15.107:0/1210028821 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70016040 con 0x562a983cfd60 2024-03-11T19:57:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- 172.21.15.107:0/1210028821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 msgr2=0x562a983da180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983da180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562a983daeb0 tx=0x7f2d7000f790 comp rx=0 tx=0).stop 2024-03-11T19:57:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- 172.21.15.107:0/1210028821 shutdown_connections 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983da180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 --2- 172.21.15.107:0/1210028821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- 172.21.15.107:0/1210028821 >> 172.21.15.107:0/1210028821 conn(0x562a983964b0 msgr2=0x562a983d81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- 172.21.15.107:0/1210028821 shutdown_connections 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- 172.21.15.107:0/1210028821 wait complete. 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 -- start start 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.689+0000 7f2d8be60740 1 Processor -- start 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a98442f80 con 0x562a983ceff0 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a984430f0 con 0x562a983cfd60 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a984432b0 con 0x562a983dc980 2024-03-11T19:57:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d8be60740 10 client.0 _ll_register_callbacks cb 0x7f2d70009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d82800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983e23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45238/0 (socket says 172.21.15.107:45238) 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d82800640 1 -- 172.21.15.107:0/3437966635 learned_addr learned my addr 172.21.15.107:0/3437966635 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d81400640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.690+0000 7f2d82800640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 -- 172.21.15.107:0/3437966635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 msgr2=0x562a983e2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983e2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 -- 172.21.15.107:0/3437966635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 msgr2=0x562a983e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 -- 172.21.15.107:0/3437966635 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a98333c50 con 0x562a983cfd60 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d82800640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d81400640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983e28f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x562a983cac20 tx=0x7f2d70012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c4070bd64525061 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7003a070 con 0x562a983cfd60 2024-03-11T19:57:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2d7001a960 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a9835ad30 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d70016050 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.691+0000 7f2d8be60740 10 client.25817 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562a982f62c0 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562a9835ad30 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f2d7003d020 con 0x562a983cfd60 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 10 client.25817 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 20 client.25817 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a98444790 0x562a98446bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 4 client.25817 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a98035600 con 0x562a98444790 2024-03-11T19:57:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.692+0000 7f2d82800640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a98444790 0x562a98446bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.693+0000 7f2d82800640 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a98444790 0x562a98446bb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d818bf0f20ef9722 server_cookie=4d9fa62a222edc5c in_seq=0 out_seq=0 2024-03-11T19:57:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.693+0000 7f2d63e00640 10 client.25817 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.782+0000 7f2d63e00640 1 -- 172.21.15.107:0/3437966635 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a98035600 con 0x562a98444790 2024-03-11T19:57:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.782+0000 7f2d63e00640 10 client.25817 handle_command_reply: tid=0 2024-03-11T19:57:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.782+0000 7f2d8be60740 1 client.25817 shutdown 2024-03-11T19:57:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.785+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 msgr2=0x562a983e28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.785+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983e28f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x562a983cac20 tx=0x7f2d70012eb0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 shutdown_connections 2024-03-11T19:57:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x562a98444790 0x562a98446bb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a983cfd60 0x562a983e28f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562a983dc980 0x562a983e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 --2- 172.21.15.107:0/3437966635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a983ceff0 0x562a983e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 >> 172.21.15.107:0/3437966635 conn(0x562a983964b0 msgr2=0x562a983c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.786+0000 7f2d63e00640 0 client.25817 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.787+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 shutdown_connections 2024-03-11T19:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.787+0000 7f2d8be60740 1 -- 172.21.15.107:0/3437966635 wait complete. 2024-03-11T19:57:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.789+0000 7f2d8be60740 20 client.25817 trim_cache size 0 max 16384 2024-03-11T19:57:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:57:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:57:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:57:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 msgr2=0x7f2d680a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 0x7f2d680a4050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d740ecaa0 tx=0x7f2d740f50e0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 msgr2=0x7f2d84197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 0x7f2d84197700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d74037040 tx=0x7f2d740045b0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 shutdown_connections 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f2d680a1b90 0x7f2d680a4050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d841062c0 0x7f2d84197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8419bfc0 0x7f2d8419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 --2- 172.21.15.107:0/140744644 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d841011f0 0x7f2d84197700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.790+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 >> 172.21.15.107:0/140744644 conn(0x7f2d840fca90 msgr2=0x7f2d840fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.791+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 shutdown_connections 2024-03-11T19:57:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:26.792+0000 7f2d83200640 1 -- 172.21.15.107:0/140744644 wait complete. 2024-03-11T19:57:27.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:26 smithi122 ceph-mon[36397]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 253 op/s 2024-03-11T19:57:27.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:26 smithi145 ceph-mon[36730]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 253 op/s 2024-03-11T19:57:27.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:26 smithi107 ceph-mon[31712]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 253 op/s 2024-03-11T19:57:29.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:28 smithi122 ceph-mon[36397]: pgmap v124: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-03-11T19:57:29.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:28 smithi145 ceph-mon[36730]: pgmap v124: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-03-11T19:57:29.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:28 smithi107 ceph-mon[31712]: pgmap v124: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-03-11T19:57:31.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:30 smithi122 ceph-mon[36397]: pgmap v125: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 953 KiB/s wr, 91 op/s 2024-03-11T19:57:31.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:30 smithi145 ceph-mon[36730]: pgmap v125: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 953 KiB/s wr, 91 op/s 2024-03-11T19:57:31.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:30 smithi107 ceph-mon[31712]: pgmap v125: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 953 KiB/s wr, 91 op/s 2024-03-11T19:57:33.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:32 smithi122 ceph-mon[36397]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-11T19:57:33.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:32 smithi145 ceph-mon[36730]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-11T19:57:33.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:32 smithi107 ceph-mon[31712]: pgmap v126: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-03-11T19:57:35.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:34 smithi122 ceph-mon[36397]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 770 KiB/s wr, 147 op/s 2024-03-11T19:57:35.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:34 smithi145 ceph-mon[36730]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 770 KiB/s wr, 147 op/s 2024-03-11T19:57:35.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:34 smithi107 ceph-mon[31712]: pgmap v127: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 770 KiB/s wr, 147 op/s 2024-03-11T19:57:37.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:36 smithi122 ceph-mon[36397]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 656 KiB/s wr, 139 op/s 2024-03-11T19:57:37.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:36 smithi145 ceph-mon[36730]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 656 KiB/s wr, 139 op/s 2024-03-11T19:57:37.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:36 smithi107 ceph-mon[31712]: pgmap v128: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 656 KiB/s wr, 139 op/s 2024-03-11T19:57:39.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:38 smithi122 ceph-mon[36397]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-11T19:57:39.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:38 smithi145 ceph-mon[36730]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-11T19:57:39.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:38 smithi107 ceph-mon[31712]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 930 KiB/s wr, 192 op/s 2024-03-11T19:57:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 -- 172.21.15.107:0/279292783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2001023f0 msgr2=0x7fe2001027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 --2- 172.21.15.107:0/279292783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2001023f0 0x7fe2001027d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402bd60 comp rx=0 tx=0).stop 2024-03-11T19:57:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 -- 172.21.15.107:0/279292783 shutdown_connections 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 --2- 172.21.15.107:0/279292783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe200102d10 0x7fe20010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 --2- 172.21.15.107:0/279292783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 --2- 172.21.15.107:0/279292783 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2001023f0 0x7fe2001027d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 -- 172.21.15.107:0/279292783 >> 172.21.15.107:0/279292783 conn(0x7fe2000771a0 msgr2=0x7fe2000775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 -- 172.21.15.107:0/279292783 shutdown_connections 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.964+0000 7fe207e00640 1 -- 172.21.15.107:0/279292783 wait complete. 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 Processor -- start 2024-03-11T19:57:39.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 -- start start 2024-03-11T19:57:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe200102d10 0x7fe200197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019c600 con 0x7fe20010b730 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019c770 con 0x7fe2001023f0 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.966+0000 7fe207e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019c930 con 0x7fe200102d10 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe204a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe200102d10 0x7fe200197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43818/0 (socket says 172.21.15.107:43818) 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205400640 1 -- 172.21.15.107:0/1041560930 learned_addr learned my addr 172.21.15.107:0/1041560930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 -- 172.21.15.107:0/1041560930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe200102d10 msgr2=0x7fe200197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe200102d10 0x7fe200197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 -- 172.21.15.107:0/1041560930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 msgr2=0x7fe200197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205e00640 1 -- 172.21.15.107:0/1041560930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009560 con 0x7fe20010b730 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.967+0000 7fe205400640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:57:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe205e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20019c000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009fd0 tx=0x7fe1f800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe1fe000640 1 -- 172.21.15.107:0/1041560930 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800ce20 con 0x7fe20010b730 2024-03-11T19:57:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe1fe000640 1 -- 172.21.15.107:0/1041560930 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f8005020 con 0x7fe20010b730 2024-03-11T19:57:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20019cc10 con 0x7fe20010b730 2024-03-11T19:57:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe1fe000640 1 -- 172.21.15.107:0/1041560930 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8010690 con 0x7fe20010b730 2024-03-11T19:57:39.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.968+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2001a4490 con 0x7fe20010b730 2024-03-11T19:57:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.972+0000 7fe20ab90740 1 Processor -- start 2024-03-11T19:57:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.973+0000 7fe1fe000640 1 -- 172.21.15.107:0/1041560930 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe1f8005500 con 0x7fe20010b730 2024-03-11T19:57:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.973+0000 7fe20ab90740 1 -- start start 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.973+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec6db0 0x55b087ec7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe20ab90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087e82620 con 0x55b087ec6db0 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe20ab90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087e348c0 con 0x55b087ed0330 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe20ab90740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087e31df0 con 0x55b087ec7b20 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe204a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe204a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59150/0 (socket says 172.21.15.107:59150) 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe204a00640 1 -- 172.21.15.107:0/2510440271 learned_addr learned my addr 172.21.15.107:0/2510440271 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe205400640 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec6db0 0x55b087ec7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.974+0000 7fe204a00640 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 -- 172.21.15.107:0/2510440271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 msgr2=0x55b087ed2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 -- 172.21.15.107:0/2510440271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec6db0 msgr2=0x55b087ec7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec6db0 0x55b087ec7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 -- 172.21.15.107:0/2510440271 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b087daeb00 con 0x55b087ec7b20 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe204a00640 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009c10 tx=0x7fe1f0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a239f30e8c3cc48 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe1ee600640 1 -- 172.21.15.107:0/2510440271 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0009d00 con 0x55b087ec7b20 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe1ee600640 1 -- 172.21.15.107:0/2510440271 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f0004500 con 0x55b087ec7b20 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.975+0000 7fe1ee600640 1 -- 172.21.15.107:0/2510440271 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0016b50 con 0x55b087ec7b20 2024-03-11T19:57:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe1fe000640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 0x7fe1e80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe1fe000640 1 -- 172.21.15.107:0/1041560930 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe1f8014070 con 0x7fe20010b730 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe205400640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 0x7fe1e80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 -- 172.21.15.107:0/2510440271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 msgr2=0x55b087ec6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009c10 tx=0x7fe1f0005850 comp rx=0 tx=0).stop 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 -- 172.21.15.107:0/2510440271 shutdown_connections 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec6db0 0x55b087ec7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2510440271 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ec6870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.976+0000 7fe20ab90740 1 -- 172.21.15.107:0/2510440271 >> 172.21.15.107:0/2510440271 conn(0x55b087e8e4b0 msgr2=0x55b087ec1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.977+0000 7fe205400640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 0x7fe1e80a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4000c00 tx=0x7fe1f40058d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.977+0000 7fe20ab90740 1 -- 172.21.15.107:0/2510440271 shutdown_connections 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.977+0000 7fe20ab90740 1 -- 172.21.15.107:0/2510440271 wait complete. 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.978+0000 7fe20ab90740 1 -- start start 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.978+0000 7fe20ab90740 1 Processor -- start 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.978+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.978+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ecd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.978+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe20ab90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087e2d5d0 con 0x55b087ed0330 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe20ab90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087dd2b50 con 0x55b087ec6db0 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe20ab90740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087dd27f0 con 0x55b087ec7b20 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe20ab90740 10 client.0 _ll_register_callbacks cb 0x55b087de6550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43834/0 (socket says 172.21.15.107:43834) 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 -- 172.21.15.107:0/3336170744 learned_addr learned my addr 172.21.15.107:0/3336170744 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe204a00640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ecd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205e00640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 -- 172.21.15.107:0/3336170744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 msgr2=0x55b087ecd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ecd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 -- 172.21.15.107:0/3336170744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 msgr2=0x55b087ecde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.979+0000 7fe205400640 1 -- 172.21.15.107:0/3336170744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f001f040 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe205400640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009430 tx=0x7fe1f4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0999070a150145b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe1e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4004450 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b087dffcd0 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe1f4005370 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f4005530 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe20ab90740 10 client.25829 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.980+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b087e2bc50 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b087dffcd0 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe1f4048040 con 0x55b087ec6db0 2024-03-11T19:57:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 10 client.25829 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 20 client.25829 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087ee0460 0x55b087ee28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 4 client.25829 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b087e23ce0 con 0x55b087ee0460 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.981+0000 7fe204a00640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087ee0460 0x55b087ee28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.982+0000 7fe204a00640 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087ee0460 0x55b087ee28a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e101f8f50d1eb3f server_cookie=d4148af85110bf5d in_seq=0 out_seq=0 2024-03-11T19:57:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.982+0000 7fe1e2a00640 10 client.25829 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe1e2a00640 1 -- 172.21.15.107:0/3336170744 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b087e23ce0 con 0x55b087ee0460 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe1e2a00640 10 client.25829 handle_command_reply: tid=0 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 client.25829 shutdown 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 msgr2=0x55b087ecd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009430 tx=0x7fe1f4009260 comp rx=0 tx=0).stop 2024-03-11T19:57:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 shutdown_connections 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087ee0460 0x55b087ee28a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec7b20 0x55b087ecd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe1e2a00640 0 client.25829 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 --2- 172.21.15.107:0/3336170744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec6db0 0x55b087ecd380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 >> 172.21.15.107:0/3336170744 conn(0x55b087e8e4b0 msgr2=0x55b087ec37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.983+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 shutdown_connections 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.984+0000 7fe20ab90740 1 -- 172.21.15.107:0/3336170744 wait complete. 2024-03-11T19:57:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:39.985+0000 7fe20ab90740 20 client.25829 trim_cache size 0 max 16384 2024-03-11T19:57:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.005+0000 7fe20ab90740 1 Processor -- start 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.005+0000 7fe20ab90740 1 -- start start 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ecfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087b75e60 con 0x55b087ed0330 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087edc170 con 0x55b087ec7b20 2024-03-11T19:57:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe20ab90740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087ebd7a0 con 0x55b087ec6db0 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe204a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe205e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe205400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ecfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe204a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:43838/0 (socket says 172.21.15.107:43838) 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.006+0000 7fe204a00640 1 -- 172.21.15.107:0/873066085 learned_addr learned my addr 172.21.15.107:0/873066085 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 -- 172.21.15.107:0/873066085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 msgr2=0x55b087ecfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ecfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 -- 172.21.15.107:0/873066085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 msgr2=0x55b087ecc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 -- 172.21.15.107:0/873066085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b087dc8530 con 0x55b087ec7b20 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe204a00640 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009c10 tx=0x7fe1f0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6808fb502b344815 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe1edc00640 1 -- 172.21.15.107:0/873066085 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000fc20 con 0x55b087ec7b20 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe1edc00640 1 -- 172.21.15.107:0/873066085 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f002b820 con 0x55b087ec7b20 2024-03-11T19:57:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.007+0000 7fe1edc00640 1 -- 172.21.15.107:0/873066085 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f002bb70 con 0x55b087ec7b20 2024-03-11T19:57:40.009 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:39 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 -- 172.21.15.107:0/873066085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 msgr2=0x55b087ecbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009c10 tx=0x7fe1f0005f90 comp rx=0 tx=0).stop 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 -- 172.21.15.107:0/873066085 shutdown_connections 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ec7b20 0x55b087ecbc70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ed0330 0x55b087ecc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 --2- 172.21.15.107:0/873066085 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ecfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.008+0000 7fe20ab90740 1 -- 172.21.15.107:0/873066085 >> 172.21.15.107:0/873066085 conn(0x55b087e8e4b0 msgr2=0x55b087ec15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.009+0000 7fe20ab90740 1 -- 172.21.15.107:0/873066085 shutdown_connections 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.009+0000 7fe20ab90740 1 -- 172.21.15.107:0/873066085 wait complete. 2024-03-11T19:57:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.009+0000 7fe20ab90740 1 -- start start 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.009+0000 7fe20ab90740 1 Processor -- start 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec7b20 0x55b087ed6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087f369d0 con 0x55b087ec7b20 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087f36b10 con 0x55b087ed0330 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087f36cd0 con 0x55b087ec6db0 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe20ab90740 10 client.0 _ll_register_callbacks cb 0x7fe1f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe205400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe205400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59180/0 (socket says 172.21.15.107:59180) 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.010+0000 7fe205400640 1 -- 172.21.15.107:0/2404970016 learned_addr learned my addr 172.21.15.107:0/2404970016 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe204a00640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec7b20 0x55b087ed6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 -- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 msgr2=0x55b087ed6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 -- 172.21.15.107:0/2404970016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec7b20 msgr2=0x55b087ed6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec7b20 0x55b087ed6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 -- 172.21.15.107:0/2404970016 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b087daeb00 con 0x55b087ec6db0 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe205400640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40040e0 tx=0x7fe1f40344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34263c2aa4118f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe1e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4034e40 con 0x55b087ec6db0 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b087e52d30 con 0x55b087ec6db0 2024-03-11T19:57:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.011+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe1f40051c0 con 0x55b087ec6db0 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f4005380 con 0x55b087ec6db0 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 10 client.25831 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b087dee2c0 con 0x55b087ec6db0 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b087e52d30 con 0x55b087ec6db0 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe1f4048040 con 0x55b087ec6db0 2024-03-11T19:57:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 10 client.25831 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 20 client.25831 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087f382d0 0x55b087f3a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 4 client.25831 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.012+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b087b2d600 con 0x55b087f382d0 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.013+0000 7fe204a00640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087f382d0 0x55b087f3a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.013+0000 7fe1e2a00640 10 client.25831 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.013+0000 7fe204a00640 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087f382d0 0x55b087f3a6f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0dfcbfa261b102 server_cookie=de425edbb5bd7411 in_seq=0 out_seq=0 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.031+0000 7fe1e2a00640 1 -- 172.21.15.107:0/2404970016 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b087b2d600 con 0x55b087f382d0 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.031+0000 7fe1e2a00640 10 client.25831 handle_command_reply: tid=0 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 client.25831 shutdown 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 msgr2=0x55b087ed5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40040e0 tx=0x7fe1f40344c0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 shutdown_connections 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55b087f382d0 0x55b087f3a6f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b087ed0330 0x55b087ed6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b087ec7b20 0x55b087ed6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 --2- 172.21.15.107:0/2404970016 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b087ec6db0 0x55b087ed5c10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe1e2a00640 0 client.25831 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 >> 172.21.15.107:0/2404970016 conn(0x55b087e8e4b0 msgr2=0x55b087ecbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 shutdown_connections 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 1 -- 172.21.15.107:0/2404970016 wait complete. 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.032+0000 7fe20ab90740 20 client.25831 trim_cache size 0 max 16384 2024-03-11T19:57:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:57:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:57:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:57:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:57:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 msgr2=0x7fe1e80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 0x7fe1e80a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4000c00 tx=0x7fe1f40058d0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 msgr2=0x7fe20019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20019c000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009fd0 tx=0x7fe1f800ef10 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 shutdown_connections 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe1e80a1cc0 0x7fe1e80a4180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20010b730 0x7fe20019c000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe200102d10 0x7fe200197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 --2- 172.21.15.107:0/1041560930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2001023f0 0x7fe200197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.033+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 >> 172.21.15.107:0/1041560930 conn(0x7fe2000771a0 msgr2=0x7fe2000fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.034+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 shutdown_connections 2024-03-11T19:57:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:40.034+0000 7fe207e00640 1 -- 172.21.15.107:0/1041560930 wait complete. 2024-03-11T19:57:40.036 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:39 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:57:40.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:39 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:57:41.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:40 smithi122 ceph-mon[36397]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 586 KiB/s wr, 129 op/s 2024-03-11T19:57:41.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:40 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:57:41.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:40 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:57:41.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:40 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:57:41.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:40 smithi145 ceph-mon[36730]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 586 KiB/s wr, 129 op/s 2024-03-11T19:57:41.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:40 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:57:41.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:40 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:57:41.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:40 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:57:41.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:40 smithi107 ceph-mon[31712]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 586 KiB/s wr, 129 op/s 2024-03-11T19:57:41.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:40 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:57:41.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:40 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:57:41.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:40 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:57:43.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:42 smithi122 ceph-mon[36397]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 923 KiB/s wr, 175 op/s 2024-03-11T19:57:43.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:42 smithi145 ceph-mon[36730]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 923 KiB/s wr, 175 op/s 2024-03-11T19:57:43.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:42 smithi107 ceph-mon[31712]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 923 KiB/s wr, 175 op/s 2024-03-11T19:57:45.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:44 smithi122 ceph-mon[36397]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 770 KiB/s wr, 131 op/s 2024-03-11T19:57:45.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:44 smithi145 ceph-mon[36730]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 770 KiB/s wr, 131 op/s 2024-03-11T19:57:45.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:44 smithi107 ceph-mon[31712]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 770 KiB/s wr, 131 op/s 2024-03-11T19:57:47.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:46 smithi122 ceph-mon[36397]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 622 KiB/s wr, 116 op/s 2024-03-11T19:57:47.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:46 smithi145 ceph-mon[36730]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 622 KiB/s wr, 116 op/s 2024-03-11T19:57:47.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:46 smithi107 ceph-mon[31712]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 622 KiB/s wr, 116 op/s 2024-03-11T19:57:49.008 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:48 smithi122 ceph-mon[36397]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-11T19:57:49.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:48 smithi145 ceph-mon[36730]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-11T19:57:49.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:48 smithi107 ceph-mon[31712]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-03-11T19:57:51.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:50 smithi122 ceph-mon[36397]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 913 KiB/s wr, 115 op/s 2024-03-11T19:57:51.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:50 smithi145 ceph-mon[36730]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 913 KiB/s wr, 115 op/s 2024-03-11T19:57:51.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:50 smithi107 ceph-mon[31712]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 913 KiB/s wr, 115 op/s 2024-03-11T19:57:53.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:52 smithi122 ceph-mon[36397]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1005 KiB/s wr, 176 op/s 2024-03-11T19:57:53.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:52 smithi145 ceph-mon[36730]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1005 KiB/s wr, 176 op/s 2024-03-11T19:57:53.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:52 smithi107 ceph-mon[31712]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1005 KiB/s wr, 176 op/s 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 -- 172.21.15.107:0/379659747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a4103db0 msgr2=0x7f72a4107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 --2- 172.21.15.107:0/379659747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a4103db0 0x7f72a4107df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7290009930 tx=0x7f729002bc30 comp rx=0 tx=0).stop 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 -- 172.21.15.107:0/379659747 shutdown_connections 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 --2- 172.21.15.107:0/379659747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41084d0 0x7f72a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 --2- 172.21.15.107:0/379659747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a4103db0 0x7f72a4107df0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 --2- 172.21.15.107:0/379659747 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a41037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 -- 172.21.15.107:0/379659747 >> 172.21.15.107:0/379659747 conn(0x7f72a40fec60 msgr2=0x7f72a4101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 -- 172.21.15.107:0/379659747 shutdown_connections 2024-03-11T19:57:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.181+0000 7f72aae00640 1 -- 172.21.15.107:0/379659747 wait complete. 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.182+0000 7f72aae00640 1 Processor -- start 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.182+0000 7f72aae00640 1 -- start start 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a41084d0 0x7f72a419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41996d0 0x7f72a4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419a1a0 con 0x7f72a41084d0 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419a310 con 0x7f72a41996d0 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419a4d0 con 0x7f72a4103400 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72a9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a41084d0 0x7f72a419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72a9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45924/0 (socket says 172.21.15.107:45924) 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72aa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41996d0 0x7f72a4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.183+0000 7f72a9a00640 1 -- 172.21.15.107:0/359021789 learned_addr learned my addr 172.21.15.107:0/359021789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 -- 172.21.15.107:0/359021789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41996d0 msgr2=0x7f72a4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41996d0 0x7f72a4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 -- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a41084d0 msgr2=0x7f72a419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a41084d0 0x7f72a419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 -- 172.21.15.107:0/359021789 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290009560 con 0x7f72a4103400 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a9a00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9f0 tx=0x7f729800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a2000640 1 -- 172.21.15.107:0/359021789 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009700 con 0x7f72a4103400 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a2000640 1 -- 172.21.15.107:0/359021789 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f729800cd80 con 0x7f72a4103400 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a41a6420 con 0x7f72a4103400 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.184+0000 7f72a2000640 1 -- 172.21.15.107:0/359021789 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298018920 con 0x7f72a4103400 2024-03-11T19:57:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.185+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a6900 con 0x7f72a4103400 2024-03-11T19:57:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 Processor -- start 2024-03-11T19:57:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 -- start start 2024-03-11T19:57:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143148ff0 0x556143149470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x556143154960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556143104620 con 0x556143148ff0 2024-03-11T19:57:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561430b68c0 con 0x556143149d60 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.188+0000 7f72af20d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561430b3df0 con 0x556143152570 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:38522/0 (socket says 172.21.15.107:38522) 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 -- 172.21.15.107:0/2637858955 learned_addr learned my addr 172.21.15.107:0/2637858955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72a9000640 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x556143154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 -- 172.21.15.107:0/2637858955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 msgr2=0x556143154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x556143154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 -- 172.21.15.107:0/2637858955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143148ff0 msgr2=0x556143149470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143148ff0 0x556143149470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72a2000640 1 -- 172.21.15.107:0/359021789 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7298018a80 con 0x7f72a4103400 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 -- 172.21.15.107:0/2637858955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556143030b00 con 0x556143149d60 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.189+0000 7f72aa400640 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f729c009f50 tx=0x7f729c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c62217f8b9828b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.190+0000 7f7296a00640 1 -- 172.21.15.107:0/2637858955 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c004d20 con 0x556143149d60 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.190+0000 7f7296a00640 1 -- 172.21.15.107:0/2637858955 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f729c004e80 con 0x556143149d60 2024-03-11T19:57:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.190+0000 7f7296a00640 1 -- 172.21.15.107:0/2637858955 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c005280 con 0x556143149d60 2024-03-11T19:57:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.190+0000 7f72af20d740 1 -- 172.21.15.107:0/2637858955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 msgr2=0x556143148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.190+0000 7f72af20d740 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f729c009f50 tx=0x7f729c0118d0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 -- 172.21.15.107:0/2637858955 shutdown_connections 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x556143154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143148ff0 0x556143149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 --2- 172.21.15.107:0/2637858955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143149d60 0x556143148ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 -- 172.21.15.107:0/2637858955 >> 172.21.15.107:0/2637858955 conn(0x5561431104b0 msgr2=0x556143143ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 -- 172.21.15.107:0/2637858955 shutdown_connections 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 -- 172.21.15.107:0/2637858955 wait complete. 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72a2000640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 0x7f728c0a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72a2000640 1 -- 172.21.15.107:0/359021789 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7298014070 con 0x7f72a4103400 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72a9000640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 0x7f728c0a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 -- start start 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.191+0000 7f72af20d740 1 Processor -- start 2024-03-11T19:57:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72a9000640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 0x7f728c0a3f60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f7290005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x556143150070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561430af5d0 con 0x556143149d60 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x556143054b50 con 0x556143152570 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561430547f0 con 0x556143148ff0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72af20d740 10 client.0 _ll_register_callbacks cb 0x556143068550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72a9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72a9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:45938/0 (socket says 172.21.15.107:45938) 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72a9a00640 1 -- 172.21.15.107:0/3286862856 learned_addr learned my addr 172.21.15.107:0/3286862856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.192+0000 7f72aa400640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x556143150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 -- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 msgr2=0x556143150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x556143150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 -- 172.21.15.107:0/3286862856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 msgr2=0x55614314fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 -- 172.21.15.107:0/3286862856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c009c00 con 0x556143148ff0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9000640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f729809a540 tx=0x7f729809a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8149b65b97a1d91c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f728b400640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298099c90 con 0x556143148ff0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7298099df0 con 0x556143148ff0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556143081cd0 con 0x556143148ff0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.193+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7298099fb0 con 0x556143148ff0 2024-03-11T19:57:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 10 client.25843 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5561430adc50 con 0x556143148ff0 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556143081cd0 con 0x556143148ff0 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f7298099700 con 0x556143148ff0 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 10 client.25843 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 20 client.25843 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x556143162530 0x556143164970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 4 client.25843 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561430a5ce0 con 0x556143162530 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.194+0000 7f72a9000640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x556143162530 0x556143164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.195+0000 7f728b400640 10 client.25843 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.195+0000 7f72a9000640 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x556143162530 0x556143164970 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bca11a38f717a92 server_cookie=18c94ee7ad91e224 in_seq=0 out_seq=0 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f728b400640 1 -- 172.21.15.107:0/3286862856 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561430a5ce0 con 0x556143162530 2024-03-11T19:57:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f728b400640 10 client.25843 handle_command_reply: tid=0 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 client.25843 shutdown 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 msgr2=0x55614314f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f729809a540 tx=0x7f729809a840 comp rx=0 tx=0).stop 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 shutdown_connections 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x556143162530 0x556143164970 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x556143150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 --2- 172.21.15.107:0/3286862856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x55614314f5f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f728b400640 0 client.25843 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 >> 172.21.15.107:0/3286862856 conn(0x5561431104b0 msgr2=0x556143145a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 shutdown_connections 2024-03-11T19:57:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 1 -- 172.21.15.107:0/3286862856 wait complete. 2024-03-11T19:57:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.196+0000 7f72af20d740 20 client.25843 trim_cache size 0 max 16384 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 Processor -- start 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 -- start start 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x556143151d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x556143159390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x5561431598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556142df7e60 con 0x556143149d60 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x556143081cd0 con 0x556143152570 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72af20d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614314f3e0 con 0x556143148ff0 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.211+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x556143159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x556143151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x556143159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34474/0 (socket says 172.21.15.107:34474) 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9000640 1 -- 172.21.15.107:0/3881697949 learned_addr learned my addr 172.21.15.107:0/3881697949 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72aa400640 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x5561431598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 -- 172.21.15.107:0/3881697949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 msgr2=0x5561431598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x5561431598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 -- 172.21.15.107:0/3881697949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 msgr2=0x556143159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x556143159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 -- 172.21.15.107:0/3881697949 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55614304a530 con 0x556143148ff0 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72aa400640 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x5561431598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:57:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f72a9a00640 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x556143151d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f729800c6a0 tx=0x7f729801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c6c8c5ce39589 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.212+0000 7f7296000640 1 -- 172.21.15.107:0/3881697949 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980b3140 con 0x556143148ff0 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f7296000640 1 -- 172.21.15.107:0/3881697949 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72980b32a0 con 0x556143148ff0 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f7296000640 1 -- 172.21.15.107:0/3881697949 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298099c90 con 0x556143148ff0 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 -- 172.21.15.107:0/3881697949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 msgr2=0x556143151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x556143151d20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f729800c6a0 tx=0x7f729801f000 comp rx=0 tx=0).stop 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 -- 172.21.15.107:0/3881697949 shutdown_connections 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143152570 0x5561431598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x556143159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 --2- 172.21.15.107:0/3881697949 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143148ff0 0x556143151d20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.213+0000 7f72af20d740 1 -- 172.21.15.107:0/3881697949 >> 172.21.15.107:0/3881697949 conn(0x5561431104b0 msgr2=0x556143143810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.214+0000 7f72af20d740 1 -- 172.21.15.107:0/3881697949 shutdown_connections 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.214+0000 7f72af20d740 1 -- 172.21.15.107:0/3881697949 wait complete. 2024-03-11T19:57:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.214+0000 7f72af20d740 1 -- start start 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.214+0000 7f72af20d740 1 Processor -- start 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143148ff0 0x55614314c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x55614314cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561431b8bc0 con 0x556143149d60 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561431b8d30 con 0x556143148ff0 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561431b8ef0 con 0x556143152570 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72af20d740 10 client.0 _ll_register_callbacks cb 0x7f7298010e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34482/0 (socket says 172.21.15.107:34482) 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 -- 172.21.15.107:0/180098338 learned_addr learned my addr 172.21.15.107:0/180098338 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72aa400640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x55614314cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9a00640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143148ff0 0x55614314c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 -- 172.21.15.107:0/180098338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 msgr2=0x55614314cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x55614314cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 -- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143148ff0 msgr2=0x55614314c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143148ff0 0x55614314c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72a9000640 1 -- 172.21.15.107:0/180098338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561430adc50 con 0x556143149d60 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.215+0000 7f72aa400640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x55614314cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f72a9000640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72a406a6e0 tx=0x7f7290009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceaa5b36ea12ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:57:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f728b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:57:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290051040 con 0x556143149d60 2024-03-11T19:57:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7290054be0 con 0x556143149d60 2024-03-11T19:57:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561430d4d30 con 0x556143149d60 2024-03-11T19:57:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7290054da0 con 0x556143149d60 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f72af20d740 10 client.15720 fetch_fsmap learned FSMap version 28 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.216+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5561430702c0 con 0x556143149d60 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5561430d4d30 con 0x556143149d60 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f729003c040 con 0x556143149d60 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:57:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 20 client.15720 populate_metadata read hostname 'smithi107' 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5561431ba6e0 0x5561431bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 4 client.15720 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.217+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556142daf600 con 0x5561431ba6e0 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.218+0000 7f72a9a00640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5561431ba6e0 0x5561431bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:57:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.218+0000 7f72a9a00640 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5561431ba6e0 0x5561431bcb00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b634feb1c9baedb server_cookie=ee962582cc9ca7bc in_seq=0 out_seq=0 2024-03-11T19:57:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.218+0000 7f728b400640 10 client.15720 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.245+0000 7f728b400640 1 -- 172.21.15.107:0/180098338 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556142daf600 con 0x5561431ba6e0 2024-03-11T19:57:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.245+0000 7f728b400640 10 client.15720 handle_command_reply: tid=0 2024-03-11T19:57:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.245+0000 7f72af20d740 1 client.15720 shutdown 2024-03-11T19:57:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.245+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 msgr2=0x55614314ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.245+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72a406a6e0 tx=0x7f7290009260 comp rx=0 tx=0).stop 2024-03-11T19:57:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 shutdown_connections 2024-03-11T19:57:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5561431ba6e0 0x5561431bcb00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556143152570 0x55614314cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556143149d60 0x55614314ca20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 --2- 172.21.15.107:0/180098338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556143148ff0 0x55614314c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 >> 172.21.15.107:0/180098338 conn(0x5561431104b0 msgr2=0x55614314dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f728b400640 0 client.15720 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.246+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 shutdown_connections 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.247+0000 7f72af20d740 1 -- 172.21.15.107:0/180098338 wait complete. 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.247+0000 7f72af20d740 20 client.15720 trim_cache size 0 max 16384 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:57:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:57:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:57:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:57:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 msgr2=0x7f728c0a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 0x7f728c0a3f60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f7290005fd0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 msgr2=0x7f72a419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9f0 tx=0x7f729800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 shutdown_connections 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f728c0a1aa0 0x7f728c0a3f60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72a41996d0 0x7f72a4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72a41084d0 0x7f72a419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 --2- 172.21.15.107:0/359021789 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72a4103400 0x7f72a419f5c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 >> 172.21.15.107:0/359021789 conn(0x7f72a40fec60 msgr2=0x7f72a40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:57:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 shutdown_connections 2024-03-11T19:57:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:57:53.248+0000 7f72aae00640 1 -- 172.21.15.107:0/359021789 wait complete. 2024-03-11T19:57:55.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:54 smithi122 ceph-mon[36397]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 835 KiB/s wr, 155 op/s 2024-03-11T19:57:55.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:54 smithi145 ceph-mon[36730]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 835 KiB/s wr, 155 op/s 2024-03-11T19:57:55.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:54 smithi107 ceph-mon[31712]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 835 KiB/s wr, 155 op/s 2024-03-11T19:57:57.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:56 smithi122 ceph-mon[36397]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 825 KiB/s wr, 139 op/s 2024-03-11T19:57:57.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:56 smithi145 ceph-mon[36730]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 825 KiB/s wr, 139 op/s 2024-03-11T19:57:57.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:56 smithi107 ceph-mon[31712]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 825 KiB/s wr, 139 op/s 2024-03-11T19:57:59.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:57:58 smithi122 ceph-mon[36397]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 227 op/s 2024-03-11T19:57:59.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:57:58 smithi145 ceph-mon[36730]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 227 op/s 2024-03-11T19:57:59.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:57:58 smithi107 ceph-mon[31712]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 227 op/s 2024-03-11T19:58:01.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:00 smithi122 ceph-mon[36397]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 675 KiB/s wr, 174 op/s 2024-03-11T19:58:01.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:00 smithi145 ceph-mon[36730]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 675 KiB/s wr, 174 op/s 2024-03-11T19:58:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:00 smithi107 ceph-mon[31712]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 675 KiB/s wr, 174 op/s 2024-03-11T19:58:03.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:02 smithi122 ceph-mon[36397]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-03-11T19:58:03.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:02 smithi145 ceph-mon[36730]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-03-11T19:58:03.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:02 smithi107 ceph-mon[31712]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.0 MiB/s wr, 228 op/s 2024-03-11T19:58:05.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:04 smithi122 ceph-mon[36397]: pgmap v142: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-03-11T19:58:05.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:04 smithi145 ceph-mon[36730]: pgmap v142: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-03-11T19:58:05.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:04 smithi107 ceph-mon[31712]: pgmap v142: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.386+0000 7f123c600640 1 -- 172.21.15.107:0/4012081962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 msgr2=0x7f1234073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.386+0000 7f123c600640 1 --2- 172.21.15.107:0/4012081962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f1234073910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1228009900 tx=0x7f122802bcd0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 -- 172.21.15.107:0/4012081962 shutdown_connections 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 --2- 172.21.15.107:0/4012081962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 --2- 172.21.15.107:0/4012081962 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f123410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 --2- 172.21.15.107:0/4012081962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f1234073910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 -- 172.21.15.107:0/4012081962 >> 172.21.15.107:0/4012081962 conn(0x7f12340fcc40 msgr2=0x7f12340ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 -- 172.21.15.107:0/4012081962 shutdown_connections 2024-03-11T19:58:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.387+0000 7f123c600640 1 -- 172.21.15.107:0/4012081962 wait complete. 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 Processor -- start 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 -- start start 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f1234199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234199e20 con 0x7f1234073530 2024-03-11T19:58:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234199f90 con 0x7f1234073e50 2024-03-11T19:58:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f123c600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419a150 con 0x7f123410da30 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f1239c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f1239c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50440/0 (socket says 172.21.15.107:50440) 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.388+0000 7f1239c00640 1 -- 172.21.15.107:0/1175682389 learned_addr learned my addr 172.21.15.107:0/1175682389 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f123a600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f1234199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 -- 172.21.15.107:0/1175682389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 msgr2=0x7f1234199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f1234199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 -- 172.21.15.107:0/1175682389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 msgr2=0x7f123419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 -- 172.21.15.107:0/1175682389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228009560 con 0x7f1234073530 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f123a600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f1234199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239200640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.389+0000 7f1239c00640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1228005940 tx=0x7f1228004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.390+0000 7f1232000640 1 -- 172.21.15.107:0/1175682389 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228004440 con 0x7f1234073530 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.390+0000 7f1232000640 1 -- 172.21.15.107:0/1175682389 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12280052d0 con 0x7f1234073530 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.390+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123419a3d0 con 0x7f1234073530 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.390+0000 7f1232000640 1 -- 172.21.15.107:0/1175682389 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228003da0 con 0x7f1234073530 2024-03-11T19:58:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.390+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12341a6640 con 0x7f1234073530 2024-03-11T19:58:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.393+0000 7f123f37c740 1 Processor -- start 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f1232000640 1 -- 172.21.15.107:0/1175682389 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1228005440 con 0x7f1234073530 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 -- start start 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 0x555aaa7d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d5db0 0x555aaa7d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa791620 con 0x555aaa7d5db0 2024-03-11T19:58:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7438c0 con 0x555aaa7df330 2024-03-11T19:58:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.394+0000 7f123f37c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa740df0 con 0x555aaa7d6b20 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d5db0 0x555aaa7d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f123a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 0x555aaa7d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39942/0 (socket says 172.21.15.107:39942) 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 -- 172.21.15.107:0/2320875109 learned_addr learned my addr 172.21.15.107:0/2320875109 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f123a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 0x555aaa7d5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59080/0 (socket says 172.21.15.107:59080) 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 -- 172.21.15.107:0/2320875109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 msgr2=0x555aaa7d5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 0x555aaa7d5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 -- 172.21.15.107:0/2320875109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d5db0 msgr2=0x555aaa7d6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d5db0 0x555aaa7d6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 -- 172.21.15.107:0/2320875109 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aaa6bdb00 con 0x555aaa7df330 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1232000640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 0x7f121c0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.395+0000 7f1239200640 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1220010480 tx=0x7f122000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac9b1adb43c3316 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.396+0000 7f1232000640 1 -- 172.21.15.107:0/1175682389 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f12280f03f0 con 0x7f1234073530 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.396+0000 7f123a600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 0x7f121c0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.396+0000 7f1226a00640 1 -- 172.21.15.107:0/2320875109 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011760 con 0x555aaa7df330 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.396+0000 7f1226a00640 1 -- 172.21.15.107:0/2320875109 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1220015070 con 0x555aaa7df330 2024-03-11T19:58:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.396+0000 7f1226a00640 1 -- 172.21.15.107:0/2320875109 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122001cea0 con 0x555aaa7df330 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123a600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 0x7f121c0a4100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c009680 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 -- 172.21.15.107:0/2320875109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 msgr2=0x555aaa7e1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1220010480 tx=0x7f122000bf90 comp rx=0 tx=0).stop 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 -- 172.21.15.107:0/2320875109 shutdown_connections 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e1720 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d5db0 0x555aaa7d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 --2- 172.21.15.107:0/2320875109 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d6b20 0x555aaa7d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 -- 172.21.15.107:0/2320875109 >> 172.21.15.107:0/2320875109 conn(0x555aaa79d4b0 msgr2=0x555aaa7d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 -- 172.21.15.107:0/2320875109 shutdown_connections 2024-03-11T19:58:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.397+0000 7f123f37c740 1 -- 172.21.15.107:0/2320875109 wait complete. 2024-03-11T19:58:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 -- start start 2024-03-11T19:58:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 Processor -- start 2024-03-11T19:58:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa73c5d0 con 0x555aaa7df330 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa6e1b50 con 0x555aaa7d6b20 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.398+0000 7f123f37c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa6e17f0 con 0x555aaa7d5db0 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f123f37c740 10 client.0 _ll_register_callbacks cb 0x555aaa6f5550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f123a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7dc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:59096/0 (socket says 172.21.15.107:59096) 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39954/0 (socket says 172.21.15.107:39954) 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239c00640 1 -- 172.21.15.107:0/4196880210 learned_addr learned my addr 172.21.15.107:0/4196880210 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 -- 172.21.15.107:0/4196880210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 msgr2=0x555aaa7dc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7dc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 -- 172.21.15.107:0/4196880210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 msgr2=0x555aaa7dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.399+0000 7f1239200640 1 -- 172.21.15.107:0/4196880210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122000bc70 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f1239200640 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x555aaa7d1c30 tx=0x7f122001ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea654f6618ede80d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f121b400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011970 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1220017070 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aaa70ecd0 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1220011ad0 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f123f37c740 10 client.25859 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.400+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555aaa73ac50 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555aaa70ecd0 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f122001a380 con 0x555aaa7d6b20 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 10 client.25859 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 20 client.25859 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa7ef2b0 0x555aaa7f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 4 client.25859 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:58:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aaa732ce0 con 0x555aaa7ef2b0 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.401+0000 7f1239c00640 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa7ef2b0 0x555aaa7f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.402+0000 7f1239c00640 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa7ef2b0 0x555aaa7f16d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af0fc589acbe09c server_cookie=d757ffad47491e08 in_seq=0 out_seq=0 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.402+0000 7f121b400640 10 client.25859 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f121b400640 1 -- 172.21.15.107:0/4196880210 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555aaa732ce0 con 0x555aaa7ef2b0 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f121b400640 10 client.25859 handle_command_reply: tid=0 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 client.25859 shutdown 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 msgr2=0x555aaa7dc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x555aaa7d1c30 tx=0x7f122001ca70 comp rx=0 tx=0).stop 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 shutdown_connections 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa7ef2b0 0x555aaa7f16d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7dc8d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 --2- 172.21.15.107:0/4196880210 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7dc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f121b400640 0 client.25859 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 >> 172.21.15.107:0/4196880210 conn(0x555aaa79d4b0 msgr2=0x555aaa7d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 shutdown_connections 2024-03-11T19:58:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 1 -- 172.21.15.107:0/4196880210 wait complete. 2024-03-11T19:58:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.403+0000 7f123f37c740 20 client.25859 trim_cache size 0 max 16384 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 Processor -- start 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 -- start start 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa484e60 con 0x555aaa7d6b20 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7d1aa0 con 0x555aaa7df330 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f123f37c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7c26a0 con 0x555aaa7d5db0 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.418+0000 7f1239200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50468/0 (socket says 172.21.15.107:50468) 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 -- 172.21.15.107:0/4229182825 learned_addr learned my addr 172.21.15.107:0/4229182825 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f123a600640 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 -- 172.21.15.107:0/4229182825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 msgr2=0x555aaa7deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 -- 172.21.15.107:0/4229182825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 msgr2=0x555aaa7e6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 -- 172.21.15.107:0/4229182825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aaa6d7530 con 0x555aaa7d6b20 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f123a600640 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.419+0000 7f1239200640 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f122000bc10 tx=0x7f122001db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69f4db6ced4eb02 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.420+0000 7f1226000640 1 -- 172.21.15.107:0/4229182825 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220005ed0 con 0x555aaa7d6b20 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.420+0000 7f1226000640 1 -- 172.21.15.107:0/4229182825 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1220015070 con 0x555aaa7d6b20 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.420+0000 7f1226000640 1 -- 172.21.15.107:0/4229182825 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220011770 con 0x555aaa7d6b20 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.420+0000 7f123f37c740 1 -- 172.21.15.107:0/4229182825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 msgr2=0x555aaa7dcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.420+0000 7f123f37c740 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f122000bc10 tx=0x7f122001db10 comp rx=0 tx=0).stop 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 -- 172.21.15.107:0/4229182825 shutdown_connections 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7df330 0x555aaa7e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7d6b20 0x555aaa7dcc30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 --2- 172.21.15.107:0/4229182825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 -- 172.21.15.107:0/4229182825 >> 172.21.15.107:0/4229182825 conn(0x555aaa79d4b0 msgr2=0x555aaa7dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 -- 172.21.15.107:0/4229182825 shutdown_connections 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 -- 172.21.15.107:0/4229182825 wait complete. 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 -- start start 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.421+0000 7f123f37c740 1 Processor -- start 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7e6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7d9310 con 0x555aaa7df330 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7d9480 con 0x555aaa7d6b20 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aaa7d9640 con 0x555aaa7d5db0 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123f37c740 10 client.0 _ll_register_callbacks cb 0x7f12280f3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:39964/0 (socket says 172.21.15.107:39964) 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 -- 172.21.15.107:0/1984501709 learned_addr learned my addr 172.21.15.107:0/1984501709 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239c00640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123a600640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 -- 172.21.15.107:0/1984501709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 msgr2=0x555aaa7e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7e6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 -- 172.21.15.107:0/1984501709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 msgr2=0x555aaa7d8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7d8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f1239200640 1 -- 172.21.15.107:0/1984501709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c012510 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.422+0000 7f123a600640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f1239200640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f122001a440 tx=0x7f1220005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b124e4271586d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f121b400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122001a5c0 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mon.2 v2:172.21.15.122:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1220017040 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aaa761d30 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f122001aeb0 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f123f37c740 10 client.25865 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.423+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555aaa73ac50 con 0x555aaa7d6b20 2024-03-11T19:58:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mon.2 v2:172.21.15.122:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555aaa761d30 con 0x555aaa7d6b20 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mon.2 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f122002d070 con 0x555aaa7d6b20 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 10 client.25865 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 20 client.25865 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa846680 0x555aaa848aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 4 client.25865 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555aaa43c600 con 0x555aaa846680 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f1239c00640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa846680 0x555aaa848aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.425+0000 7f121b400640 10 client.25865 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.424+0000 7f1239c00640 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa846680 0x555aaa848aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5363b0321e25eef server_cookie=c4b42eaa0ff9d566 in_seq=0 out_seq=0 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.436+0000 7f121b400640 1 -- 172.21.15.107:0/1984501709 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555aaa43c600 con 0x555aaa846680 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.436+0000 7f121b400640 10 client.25865 handle_command_reply: tid=0 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.436+0000 7f123f37c740 1 client.25865 shutdown 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.436+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 msgr2=0x555aaa7e66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.436+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f122001a440 tx=0x7f1220005870 comp rx=0 tx=0).stop 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 shutdown_connections 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x555aaa846680 0x555aaa848aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aaa7d6b20 0x555aaa7e66a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555aaa7df330 0x555aaa7d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 --2- 172.21.15.107:0/1984501709 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555aaa7d5db0 0x555aaa7e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f121b400640 0 client.25865 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 >> 172.21.15.107:0/1984501709 conn(0x555aaa79d4b0 msgr2=0x555aaa7cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 shutdown_connections 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 1 -- 172.21.15.107:0/1984501709 wait complete. 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.437+0000 7f123f37c740 20 client.25865 trim_cache size 0 max 16384 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:58:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:58:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 msgr2=0x7f121c0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 0x7f121c0a4100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c009680 comp rx=0 tx=0).stop 2024-03-11T19:58:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 msgr2=0x7f123419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1228005940 tx=0x7f1228004040 comp rx=0 tx=0).stop 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 shutdown_connections 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f121c0a1c40 0x7f121c0a4100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1234073e50 0x7f123419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f123410da30 0x7f1234199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 --2- 172.21.15.107:0/1175682389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1234073530 0x7f123419f610 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 >> 172.21.15.107:0/1175682389 conn(0x7f12340fcc40 msgr2=0x7f123410fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 shutdown_connections 2024-03-11T19:58:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:06.438+0000 7f123c600640 1 -- 172.21.15.107:0/1175682389 wait complete. 2024-03-11T19:58:07.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:06 smithi122 ceph-mon[36397]: pgmap v143: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 865 KiB/s wr, 174 op/s 2024-03-11T19:58:07.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:06 smithi145 ceph-mon[36730]: pgmap v143: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 865 KiB/s wr, 174 op/s 2024-03-11T19:58:07.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:06 smithi107 ceph-mon[31712]: pgmap v143: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 865 KiB/s wr, 174 op/s 2024-03-11T19:58:09.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:08 smithi122 ceph-mon[36397]: pgmap v144: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-11T19:58:09.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:08 smithi145 ceph-mon[36730]: pgmap v144: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-11T19:58:09.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:08 smithi107 ceph-mon[31712]: pgmap v144: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-11T19:58:11.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:10 smithi122 ceph-mon[36397]: pgmap v145: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-03-11T19:58:11.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:10 smithi145 ceph-mon[36730]: pgmap v145: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-03-11T19:58:11.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:10 smithi107 ceph-mon[31712]: pgmap v145: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.0 MiB/s wr, 148 op/s 2024-03-11T19:58:13.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:12 smithi122 ceph-mon[36397]: pgmap v146: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-03-11T19:58:13.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:12 smithi145 ceph-mon[36730]: pgmap v146: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-03-11T19:58:13.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:12 smithi107 ceph-mon[31712]: pgmap v146: 129 pgs: 129 active+clean; 2.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-03-11T19:58:15.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:14 smithi122 ceph-mon[36397]: pgmap v147: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-03-11T19:58:15.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:14 smithi145 ceph-mon[36730]: pgmap v147: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-03-11T19:58:15.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:14 smithi107 ceph-mon[31712]: pgmap v147: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2024-03-11T19:58:17.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:16 smithi122 ceph-mon[36397]: pgmap v148: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-03-11T19:58:17.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:16 smithi122 ceph-mon[36397]: from='mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935]' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:17.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:16 smithi122 ceph-mon[36397]: from='mds.? ' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:17.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:16 smithi145 ceph-mon[36730]: pgmap v148: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-03-11T19:58:17.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:16 smithi145 ceph-mon[36730]: from='mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935]' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:17.033 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:16 smithi145 ceph-mon[36730]: from='mds.? ' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:17.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:16 smithi107 ceph-mon[31712]: pgmap v148: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-03-11T19:58:17.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:16 smithi107 ceph-mon[31712]: from='mds.? [v2:172.21.15.107:6840/981878935,v1:172.21.15.107:6841/981878935]' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:17.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:16 smithi107 ceph-mon[31712]: from='mds.? ' entity='mds.a' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]: dispatch 2024-03-11T19:58:18.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:17 smithi145 ceph-mon[36730]: evicting unresponsive client smithi107:x (25421), after 300.604 seconds 2024-03-11T19:58:18.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:17 smithi145 ceph-mon[36730]: Evicting (and blocklisting) client session 25421 (172.21.15.107:0/1019473398) 2024-03-11T19:58:18.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:17 smithi145 ceph-mon[36730]: from='mds.? ' entity='mds.a' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]': finished 2024-03-11T19:58:18.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:17 smithi145 ceph-mon[36730]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:58:18.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:17 smithi107 ceph-mon[31712]: evicting unresponsive client smithi107:x (25421), after 300.604 seconds 2024-03-11T19:58:18.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:17 smithi107 ceph-mon[31712]: Evicting (and blocklisting) client session 25421 (172.21.15.107:0/1019473398) 2024-03-11T19:58:18.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:17 smithi107 ceph-mon[31712]: from='mds.? ' entity='mds.a' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]': finished 2024-03-11T19:58:18.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:17 smithi107 ceph-mon[31712]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:58:18.258 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:17 smithi122 ceph-mon[36397]: evicting unresponsive client smithi107:x (25421), after 300.604 seconds 2024-03-11T19:58:18.258 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:17 smithi122 ceph-mon[36397]: Evicting (and blocklisting) client session 25421 (172.21.15.107:0/1019473398) 2024-03-11T19:58:18.258 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:17 smithi122 ceph-mon[36397]: from='mds.? ' entity='mds.a' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.107:0/1019473398"}]': finished 2024-03-11T19:58:18.258 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:17 smithi122 ceph-mon[36397]: osdmap e97: 12 total, 12 up, 12 in 2024-03-11T19:58:19.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:18 smithi145 ceph-mon[36730]: pgmap v150: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:19.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:18 smithi107 ceph-mon[31712]: pgmap v150: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:19.158 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:18 smithi122 ceph-mon[36397]: pgmap v150: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 -- 172.21.15.107:0/601105462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e01020e0 msgr2=0x7ff6e0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 --2- 172.21.15.107:0/601105462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e01020e0 0x7ff6e0069980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009900 tx=0x7ff6d002bcc0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 -- 172.21.15.107:0/601105462 shutdown_connections 2024-03-11T19:58:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 --2- 172.21.15.107:0/601105462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e010d940 0x7ff6e010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 --2- 172.21.15.107:0/601105462 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e0069ec0 0x7ff6e006a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 --2- 172.21.15.107:0/601105462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e01020e0 0x7ff6e0069980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.583+0000 7ff6e7200640 1 -- 172.21.15.107:0/601105462 >> 172.21.15.107:0/601105462 conn(0x7ff6e0076e50 msgr2=0x7ff6e0077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.584+0000 7ff6e7200640 1 -- 172.21.15.107:0/601105462 shutdown_connections 2024-03-11T19:58:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.584+0000 7ff6e7200640 1 -- 172.21.15.107:0/601105462 wait complete. 2024-03-11T19:58:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.585+0000 7ff6e7200640 1 Processor -- start 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.585+0000 7ff6e7200640 1 -- start start 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e01020e0 0x7ff6e019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e010d940 0x7ff6e0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0199ec0 con 0x7ff6e0069ec0 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e019a030 con 0x7ff6e01020e0 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e019a1f0 con 0x7ff6e010d940 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43438/0 (socket says 172.21.15.107:43438) 2024-03-11T19:58:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 -- 172.21.15.107:0/1667507437 learned_addr learned my addr 172.21.15.107:0/1667507437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6df400640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e01020e0 0x7ff6e019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e010d940 0x7ff6e0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 -- 172.21.15.107:0/1667507437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e010d940 msgr2=0x7ff6e0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e010d940 0x7ff6e0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 -- 172.21.15.107:0/1667507437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e01020e0 msgr2=0x7ff6e019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e01020e0 0x7ff6e019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.586+0000 7ff6dfe00640 1 -- 172.21.15.107:0/1667507437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0009560 con 0x7ff6e0069ec0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.587+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d002d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.587+0000 7ff6dcc00640 1 -- 172.21.15.107:0/1667507437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002dd40 con 0x7ff6e0069ec0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.587+0000 7ff6dcc00640 1 -- 172.21.15.107:0/1667507437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6d00052d0 con 0x7ff6e0069ec0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.587+0000 7ff6dcc00640 1 -- 172.21.15.107:0/1667507437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0003c90 con 0x7ff6e0069ec0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.587+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e019a470 con 0x7ff6e0069ec0 2024-03-11T19:58:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.588+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a65f0 con 0x7ff6e0069ec0 2024-03-11T19:58:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 Processor -- start 2024-03-11T19:58:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6dcc00640 1 -- 172.21.15.107:0/1667507437 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff6d0005440 con 0x7ff6e0069ec0 2024-03-11T19:58:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 -- start start 2024-03-11T19:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e47230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784e02620 con 0x564784e46db0 2024-03-11T19:58:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784db48c0 con 0x564784e58b40 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.592+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784db1df0 con 0x564784e47b20 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e47230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6e5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6e5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57872/0 (socket says 172.21.15.107:57872) 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6e5200640 1 -- 172.21.15.107:0/1953661832 learned_addr learned my addr 172.21.15.107:0/1953661832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 -- 172.21.15.107:0/1953661832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 msgr2=0x564784e46870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 -- 172.21.15.107:0/1953661832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 msgr2=0x564784e47230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e47230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 -- 172.21.15.107:0/1953661832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564784d2eb00 con 0x564784e58b40 2024-03-11T19:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.593+0000 7ff6df400640 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5af30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc005fd0 tx=0x7ff6cc00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566e93bf9193131c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6d5600640 1 -- 172.21.15.107:0/1953661832 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc003bd0 con 0x564784e58b40 2024-03-11T19:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6d5600640 1 -- 172.21.15.107:0/1953661832 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc003d30 con 0x564784e58b40 2024-03-11T19:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6d5600640 1 -- 172.21.15.107:0/1953661832 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc004e60 con 0x564784e58b40 2024-03-11T19:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6dcc00640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 0x7ff6c40a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6dcc00640 1 -- 172.21.15.107:0/1667507437 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7ff6d00efec0 con 0x7ff6e0069ec0 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.594+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 0x7ff6c40a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1953661832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 msgr2=0x564784e5af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5af30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc005fd0 tx=0x7ff6cc00bb20 comp rx=0 tx=0).stop 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1953661832 shutdown_connections 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5af30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1953661832 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e46870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1953661832 >> 172.21.15.107:0/1953661832 conn(0x564784e0e4b0 msgr2=0x564784e41aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 0x7ff6c40a4200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005f40 tx=0x7ff6d80096a0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1953661832 shutdown_connections 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.595+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1953661832 wait complete. 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 -- start start 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 Processor -- start 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e58b40 0x564784e56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784dad5d0 con 0x564784e58b40 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784d52b50 con 0x564784e46db0 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784d527f0 con 0x564784e47b20 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.596+0000 7ff6e9ffb740 10 client.0 _ll_register_callbacks cb 0x564784d66550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6dfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6dfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:44168/0 (socket says 172.21.15.107:44168) 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6dfe00640 1 -- 172.21.15.107:0/14394557 learned_addr learned my addr 172.21.15.107:0/14394557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6e5200640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e58b40 0x564784e56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 -- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 msgr2=0x564784e55be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 -- 172.21.15.107:0/14394557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e58b40 msgr2=0x564784e56660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e58b40 0x564784e56660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 -- 172.21.15.107:0/14394557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc004290 con 0x564784e47b20 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.597+0000 7ff6df400640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e56120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x564784e42c30 tx=0x7ff6cc00bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ac0767d784c24e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6ca000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011640 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7ff6cc0117a0 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564784d7fcd0 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6cc003de0 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6e9ffb740 10 client.25876 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.598+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564784dabc50 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564784d7fcd0 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff6cc019ac0 con 0x564784e47b20 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 10 client.25876 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 20 client.25876 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784e68a80 0x564784e6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 4 client.25876 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.599+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564784da3ce0 con 0x564784e68a80 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.600+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784e68a80 0x564784e6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.600+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784e68a80 0x564784e6aec0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4609db494a3b9e3 server_cookie=5fd38c92f4174462 in_seq=0 out_seq=0 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.600+0000 7ff6ca000640 10 client.25876 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6ca000640 1 -- 172.21.15.107:0/14394557 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564784da3ce0 con 0x564784e68a80 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6ca000640 10 client.25876 handle_command_reply: tid=0 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 client.25876 shutdown 2024-03-11T19:58:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 msgr2=0x564784e56120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e56120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x564784e42c30 tx=0x7ff6cc00bfa0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 shutdown_connections 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784e68a80 0x564784e6aec0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e58b40 0x564784e56660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e47b20 0x564784e56120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/14394557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e46db0 0x564784e55be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6ca000640 0 client.25876 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 >> 172.21.15.107:0/14394557 conn(0x564784e0e4b0 msgr2=0x564784e437f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 shutdown_connections 2024-03-11T19:58:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.601+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/14394557 wait complete. 2024-03-11T19:58:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.602+0000 7ff6e9ffb740 20 client.25876 trim_cache size 0 max 16384 2024-03-11T19:58:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 Processor -- start 2024-03-11T19:58:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 -- start start 2024-03-11T19:58:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 0x564784e58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 0x564784e56480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784af5e60 con 0x564784e47b20 2024-03-11T19:58:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784e656a0 con 0x564784e58b40 2024-03-11T19:58:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.616+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784e336a0 con 0x564784e46db0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6df400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 0x564784e56480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6df400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 0x564784e56480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43472/0 (socket says 172.21.15.107:43472) 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6df400640 1 -- 172.21.15.107:0/2317976402 learned_addr learned my addr 172.21.15.107:0/2317976402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 0x564784e58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 -- 172.21.15.107:0/2317976402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 msgr2=0x564784e5f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 -- 172.21.15.107:0/2317976402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 msgr2=0x564784e56480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 0x564784e56480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.617+0000 7ff6dfe00640 1 -- 172.21.15.107:0/2317976402 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564784d48530 con 0x564784e46db0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.618+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 0x564784e58310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005ec0 tx=0x7ff6d00ef040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d146f54dd0ef42 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.618+0000 7ff6d4c00640 1 -- 172.21.15.107:0/2317976402 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00f33e0 con 0x564784e46db0 2024-03-11T19:58:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.618+0000 7ff6d4c00640 1 -- 172.21.15.107:0/2317976402 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6d00b8a60 con 0x564784e46db0 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.618+0000 7ff6d4c00640 1 -- 172.21.15.107:0/2317976402 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00bc7c0 con 0x564784e46db0 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/2317976402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 msgr2=0x564784e58310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 0x564784e58310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005ec0 tx=0x7ff6d00ef040 comp rx=0 tx=0).stop 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/2317976402 shutdown_connections 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e58b40 0x564784e5f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e47b20 0x564784e56480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/2317976402 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e46db0 0x564784e58310 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/2317976402 >> 172.21.15.107:0/2317976402 conn(0x564784e0e4b0 msgr2=0x564784e544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/2317976402 shutdown_connections 2024-03-11T19:58:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.619+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/2317976402 wait complete. 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.620+0000 7ff6e9ffb740 1 -- start start 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.620+0000 7ff6e9ffb740 1 Processor -- start 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e47b20 0x564784e5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784e5cc40 con 0x564784e46db0 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784e5cdb0 con 0x564784e47b20 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x564784ebefe0 con 0x564784e58b40 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e9ffb740 10 client.0 _ll_register_callbacks cb 0x7ff6d00f3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:57916/0 (socket says 172.21.15.107:57916) 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6e5200640 1 -- 172.21.15.107:0/1111624820 learned_addr learned my addr 172.21.15.107:0/1111624820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6df400640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e47b20 0x564784e5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.621+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 -- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e47b20 msgr2=0x564784e5c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e47b20 0x564784e5c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 -- 172.21.15.107:0/1111624820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 msgr2=0x564784e5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 -- 172.21.15.107:0/1111624820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a65f0 con 0x564784e58b40 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e5200640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801e450 tx=0x7ff6d8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b1a444586b4ed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6ca000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8024040 con 0x564784e58b40 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7ff6d8011710 con 0x564784e58b40 2024-03-11T19:58:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.622+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564784dd2d30 con 0x564784e58b40 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d8016070 con 0x564784e58b40 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 10 client.25882 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564784d6e2c0 con 0x564784e58b40 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564784dd2d30 con 0x564784e58b40 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff6d801cae0 con 0x564784e58b40 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 10 client.25882 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 20 client.25882 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784ec0810 0x564784ec2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 4 client.25882 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:58:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564784aad600 con 0x564784ec0810 2024-03-11T19:58:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.623+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784ec0810 0x564784ec2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.624+0000 7ff6dfe00640 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784ec0810 0x564784ec2c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8577cba3e1d7dc35 server_cookie=3b17196de6b1e71 in_seq=0 out_seq=0 2024-03-11T19:58:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.624+0000 7ff6ca000640 10 client.25882 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.637+0000 7ff6ca000640 1 -- 172.21.15.107:0/1111624820 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564784aad600 con 0x564784ec0810 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.637+0000 7ff6ca000640 10 client.25882 handle_command_reply: tid=0 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.637+0000 7ff6e9ffb740 1 client.25882 shutdown 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.637+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 msgr2=0x564784e5c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.637+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801e450 tx=0x7ff6d8004ab0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 shutdown_connections 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x564784ec0810 0x564784ec2c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564784e47b20 0x564784e5c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x564784e58b40 0x564784e5c560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6ca000640 0 client.25882 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 --2- 172.21.15.107:0/1111624820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564784e46db0 0x564784e5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 >> 172.21.15.107:0/1111624820 conn(0x564784e0e4b0 msgr2=0x564784e404b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 shutdown_connections 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 1 -- 172.21.15.107:0/1111624820 wait complete. 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.638+0000 7ff6e9ffb740 20 client.25882 trim_cache size 0 max 16384 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:58:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 msgr2=0x7ff6c40a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 0x7ff6c40a4200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005f40 tx=0x7ff6d80096a0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 msgr2=0x7ff6e019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d002d880 comp rx=0 tx=0).stop 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 shutdown_connections 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff6c40a1d40 0x7ff6c40a4200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6e01020e0 0x7ff6e019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6e010d940 0x7ff6e0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 --2- 172.21.15.107:0/1667507437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6e0069ec0 0x7ff6e019f670 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 >> 172.21.15.107:0/1667507437 conn(0x7ff6e0076e50 msgr2=0x7ff6e00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 shutdown_connections 2024-03-11T19:58:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:19.639+0000 7ff6e7200640 1 -- 172.21.15.107:0/1667507437 wait complete. 2024-03-11T19:58:20.257 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:19 smithi122 ceph-mon[36397]: pgmap v151: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:20.281 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:19 smithi145 ceph-mon[36730]: pgmap v151: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:20.288 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:19 smithi107 ceph-mon[31712]: pgmap v151: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 988 KiB/s wr, 201 op/s 2024-03-11T19:58:23.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:22 smithi122 ceph-mon[36397]: pgmap v152: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 766 KiB/s wr, 155 op/s 2024-03-11T19:58:23.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:22 smithi145 ceph-mon[36730]: pgmap v152: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 766 KiB/s wr, 155 op/s 2024-03-11T19:58:23.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:22 smithi107 ceph-mon[31712]: pgmap v152: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 766 KiB/s wr, 155 op/s 2024-03-11T19:58:25.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:24 smithi122 ceph-mon[36397]: pgmap v153: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:25.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:24 smithi145 ceph-mon[36730]: pgmap v153: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:25.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:24 smithi107 ceph-mon[31712]: pgmap v153: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:27.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:26 smithi122 ceph-mon[36397]: pgmap v154: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:27.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:26 smithi145 ceph-mon[36730]: pgmap v154: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:27.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:26 smithi107 ceph-mon[31712]: pgmap v154: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 684 KiB/s wr, 133 op/s 2024-03-11T19:58:29.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:28 smithi122 ceph-mon[36397]: pgmap v155: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 953 KiB/s wr, 127 op/s 2024-03-11T19:58:29.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:28 smithi145 ceph-mon[36730]: pgmap v155: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 953 KiB/s wr, 127 op/s 2024-03-11T19:58:29.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:28 smithi107 ceph-mon[31712]: pgmap v155: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 953 KiB/s wr, 127 op/s 2024-03-11T19:58:31.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:30 smithi122 ceph-mon[36397]: pgmap v156: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 572 KiB/s wr, 98 op/s 2024-03-11T19:58:31.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:30 smithi145 ceph-mon[36730]: pgmap v156: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 572 KiB/s wr, 98 op/s 2024-03-11T19:58:31.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:30 smithi107 ceph-mon[31712]: pgmap v156: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 572 KiB/s wr, 98 op/s 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 -- 172.21.15.107:0/4011002971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 msgr2=0x7fe398103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 --2- 172.21.15.107:0/4011002971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe398103980 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe388009930 tx=0x7fe38802bc20 comp rx=0 tx=0).stop 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 -- 172.21.15.107:0/4011002971 shutdown_connections 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 --2- 172.21.15.107:0/4011002971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe398108670 0x7fe398110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 --2- 172.21.15.107:0/4011002971 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398103f50 0x7fe398107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 --2- 172.21.15.107:0/4011002971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe398103980 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 -- 172.21.15.107:0/4011002971 >> 172.21.15.107:0/4011002971 conn(0x7fe3980fee00 msgr2=0x7fe398101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 -- 172.21.15.107:0/4011002971 shutdown_connections 2024-03-11T19:58:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.799+0000 7fe39e000640 1 -- 172.21.15.107:0/4011002971 wait complete. 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 Processor -- start 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 -- start start 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe39819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 0x7fe39819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 0x7fe398199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819a280 con 0x7fe3981035a0 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819a3f0 con 0x7fe3981997d0 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.800+0000 7fe39e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819a560 con 0x7fe398108670 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe39819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe396a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 0x7fe39819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 0x7fe398199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe39819f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43350/0 (socket says 172.21.15.107:43350) 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe396a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 0x7fe39819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37324/0 (socket says 172.21.15.107:37324) 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397400640 1 -- 172.21.15.107:0/4066497978 learned_addr learned my addr 172.21.15.107:0/4066497978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 -- 172.21.15.107:0/4066497978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 msgr2=0x7fe39819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 0x7fe39819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 -- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 msgr2=0x7fe39819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe39819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 -- 172.21.15.107:0/4066497978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009840 con 0x7fe3981997d0 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe397e00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 0x7fe398199c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe39000ec50 tx=0x7fe39000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.801+0000 7fe38fe00640 1 -- 172.21.15.107:0/4066497978 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390003b70 con 0x7fe3981997d0 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.802+0000 7fe38fe00640 1 -- 172.21.15.107:0/4066497978 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe39000ce70 con 0x7fe3981997d0 2024-03-11T19:58:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.802+0000 7fe38fe00640 1 -- 172.21.15.107:0/4066497978 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390010830 con 0x7fe3981997d0 2024-03-11T19:58:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.802+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe388009560 con 0x7fe3981997d0 2024-03-11T19:58:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.802+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981a6810 con 0x7fe3981997d0 2024-03-11T19:58:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.806+0000 7fe3a0c45740 1 Processor -- start 2024-03-11T19:58:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.807+0000 7fe38fe00640 1 -- 172.21.15.107:0/4066497978 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fe390010990 con 0x7fe3981997d0 2024-03-11T19:58:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 -- start start 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba99b620 con 0x55f0ba9e9330 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba94d8c0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.808+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba94adf0 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe396a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe396a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9df870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:58188/0 (socket says 172.21.15.107:58188) 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe396a00640 1 -- 172.21.15.107:0/3262784669 learned_addr learned my addr 172.21.15.107:0/3262784669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37340/0 (socket says 172.21.15.107:37340) 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe396a00640 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe38fe00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 0x7fe37c0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 -- 172.21.15.107:0/3262784669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 msgr2=0x55f0ba9df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 -- 172.21.15.107:0/3262784669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 msgr2=0x55f0ba9eb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9eb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397400640 1 -- 172.21.15.107:0/3262784669 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ba8c7b00 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe38fe00640 1 -- 172.21.15.107:0/4066497978 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fe390014070 con 0x7fe3981997d0 2024-03-11T19:58:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.809+0000 7fe397e00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 0x7fe37c0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.810+0000 7fe397400640 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe388009900 tx=0x7fe38802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e21652dfc6e00d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.810+0000 7fe397e00640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 0x7fe37c0a4100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3900cbc90 tx=0x7fe390004970 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.810+0000 7fe383e00640 1 -- 172.21.15.107:0/3262784669 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388004880 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.810+0000 7fe383e00640 1 -- 172.21.15.107:0/3262784669 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3880049e0 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.810+0000 7fe383e00640 1 -- 172.21.15.107:0/3262784669 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388005050 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3262784669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 msgr2=0x55f0ba9e0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe388009900 tx=0x7fe38802c640 comp rx=0 tx=0).stop 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3262784669 shutdown_connections 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9eb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e0230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3262784669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3262784669 >> 172.21.15.107:0/3262784669 conn(0x55f0ba9a74b0 msgr2=0x55f0ba9daaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3262784669 shutdown_connections 2024-03-11T19:58:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.811+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3262784669 wait complete. 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 -- start start 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 Processor -- start 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e9330 0x55f0ba9e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba9465d0 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba8ebb50 con 0x55f0ba9e9330 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.812+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba8eb7f0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe3a0c45740 10 client.0 _ll_register_callbacks cb 0x55f0ba8ff550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37350/0 (socket says 172.21.15.107:37350) 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 -- 172.21.15.107:0/3353401368 learned_addr learned my addr 172.21.15.107:0/3353401368 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe396a00640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe396a00640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e9330 0x55f0ba9e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 -- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e9330 msgr2=0x55f0ba9e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e9330 0x55f0ba9e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 -- 172.21.15.107:0/3353401368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 msgr2=0x55f0ba9e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe397400640 1 -- 172.21.15.107:0/3353401368 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe398110a50 con 0x55f0ba9e0b20 2024-03-11T19:58:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.813+0000 7fe396a00640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe397400640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe38802ca80 tx=0x7fe38802cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e6c27f0c3e9e88 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe380c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388005200 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0ba918cd0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fe38804d070 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe388005360 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe3a0c45740 10 client.25894 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.814+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f0ba944c50 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f0ba918cd0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe38803c050 con 0x55f0ba9e0b20 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 10 client.25894 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 20 client.25894 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0ba9fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 4 client.25894 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:58:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0ba93cce0 con 0x55f0ba9f8fc0 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.815+0000 7fe396a00640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0ba9fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.816+0000 7fe396a00640 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0ba9fb3e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d915d38c6b937b47 server_cookie=6f761eda29f03397 in_seq=0 out_seq=0 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.816+0000 7fe380c00640 10 client.25894 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe380c00640 1 -- 172.21.15.107:0/3353401368 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0ba93cce0 con 0x55f0ba9f8fc0 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe380c00640 10 client.25894 handle_command_reply: tid=0 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 client.25894 shutdown 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 msgr2=0x55f0ba9e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe38802ca80 tx=0x7fe38802cab0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 shutdown_connections 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0ba9fb3e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e9330 0x55f0ba9e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe380c00640 0 client.25894 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e68e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3353401368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.817+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 >> 172.21.15.107:0/3353401368 conn(0x55f0ba9a74b0 msgr2=0x55f0ba9dc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.818+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 shutdown_connections 2024-03-11T19:58:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.818+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3353401368 wait complete. 2024-03-11T19:58:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.818+0000 7fe3a0c45740 20 client.25894 trim_cache size 0 max 16384 2024-03-11T19:58:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.832+0000 7fe3a0c45740 1 Processor -- start 2024-03-11T19:58:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 -- start start 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba68ee60 con 0x55f0ba9e9330 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba9db8a0 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba9f5e20 con 0x55f0ba9e0b20 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe396a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe396a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:58198/0 (socket says 172.21.15.107:58198) 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe396a00640 1 -- 172.21.15.107:0/3973036352 learned_addr learned my addr 172.21.15.107:0/3973036352 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe397400640 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.833+0000 7fe396a00640 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 -- 172.21.15.107:0/3973036352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 msgr2=0x55f0ba9e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 -- 172.21.15.107:0/3973036352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 msgr2=0x55f0ba9f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 -- 172.21.15.107:0/3973036352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ba8e1530 con 0x55f0ba9e0b20 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe397400640 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e6c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe38802bc60 tx=0x7fe388004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333224f7756a0ddb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe383400640 1 -- 172.21.15.107:0/3973036352 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880046c0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe383400640 1 -- 172.21.15.107:0/3973036352 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3880044c0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.834+0000 7fe383400640 1 -- 172.21.15.107:0/3973036352 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388005550 con 0x55f0ba9e0b20 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3973036352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 msgr2=0x55f0ba9e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e6c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe38802bc60 tx=0x7fe388004c10 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3973036352 shutdown_connections 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0ba9f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9e6c40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/3973036352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3973036352 >> 172.21.15.107:0/3973036352 conn(0x55f0ba9a74b0 msgr2=0x55f0ba9da5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3973036352 shutdown_connections 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- 172.21.15.107:0/3973036352 wait complete. 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.835+0000 7fe3a0c45740 1 -- start start 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 Processor -- start 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0baa4e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba9ecf20 con 0x55f0ba9e9330 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0baa4e9d0 con 0x55f0ba9e0b20 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0baa4eb40 con 0x55f0ba9dfdb0 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe3a0c45740 10 client.0 _ll_register_callbacks cb 0x7fe384009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.836+0000 7fe396a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9ec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:37362/0 (socket says 172.21.15.107:37362) 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 -- 172.21.15.107:0/2817299939 learned_addr learned my addr 172.21.15.107:0/2817299939 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe397400640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0baa4e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 -- 172.21.15.107:0/2817299939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 msgr2=0x55f0ba9ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 -- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 msgr2=0x55f0ba9ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 -- 172.21.15.107:0/2817299939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0ba944c50 con 0x55f0ba9e9330 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe396a00640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0baa4e490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe384004320 tx=0x7fe384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1fbe710ba2a52d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.837+0000 7fe380c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384028070 con 0x55f0ba9e9330 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0ba96bd30 con 0x55f0ba9e9330 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fe384005020 con 0x55f0ba9e9330 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe384016040 con 0x55f0ba9e9330 2024-03-11T19:58:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe3a0c45740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f0ba9072c0 con 0x55f0ba9e9330 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.838+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f0ba96bd30 con 0x55f0ba9e9330 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe384032050 con 0x55f0ba9e9330 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 20 client.15771 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0baa52400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 4 client.15771 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.839+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0ba646600 con 0x55f0ba9f8fc0 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.840+0000 7fe397400640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0baa52400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.840+0000 7fe397400640 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0baa52400 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b13514fb6362aa server_cookie=6bb4576eeae0c9cf in_seq=0 out_seq=0 2024-03-11T19:58:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.840+0000 7fe380c00640 10 client.15771 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.854+0000 7fe380c00640 1 -- 172.21.15.107:0/2817299939 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0ba646600 con 0x55f0ba9f8fc0 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.854+0000 7fe380c00640 10 client.15771 handle_command_reply: tid=0 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.854+0000 7fe3a0c45740 1 client.15771 shutdown 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 msgr2=0x55f0baa4e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0baa4e490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe384004320 tx=0x7fe384004730 comp rx=0 tx=0).stop 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 shutdown_connections 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x55f0ba9f8fc0 0x55f0baa52400 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0ba9e0b20 0x55f0ba9ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f0ba9e9330 0x55f0baa4e490 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe380c00640 0 client.15771 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 --2- 172.21.15.107:0/2817299939 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55f0ba9dfdb0 0x55f0ba9ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 >> 172.21.15.107:0/2817299939 conn(0x55f0ba9a74b0 msgr2=0x55f0ba9e4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 shutdown_connections 2024-03-11T19:58:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 1 -- 172.21.15.107:0/2817299939 wait complete. 2024-03-11T19:58:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.855+0000 7fe3a0c45740 20 client.15771 trim_cache size 0 max 16384 2024-03-11T19:58:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:58:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:58:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:58:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 msgr2=0x7fe37c0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 0x7fe37c0a4100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe3900cbc90 tx=0x7fe390004970 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 msgr2=0x7fe398199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 0x7fe398199c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe39000ec50 tx=0x7fe39000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 shutdown_connections 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fe37c0a1c40 0x7fe37c0a4100 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3981997d0 0x7fe398199c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe398108670 0x7fe39819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 --2- 172.21.15.107:0/4066497978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3981035a0 0x7fe39819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 >> 172.21.15.107:0/4066497978 conn(0x7fe3980fee00 msgr2=0x7fe3981011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 shutdown_connections 2024-03-11T19:58:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:32.856+0000 7fe39e000640 1 -- 172.21.15.107:0/4066497978 wait complete. 2024-03-11T19:58:33.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:32 smithi122 ceph-mon[36397]: pgmap v157: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 755 KiB/s wr, 138 op/s 2024-03-11T19:58:33.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:32 smithi145 ceph-mon[36730]: pgmap v157: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 755 KiB/s wr, 138 op/s 2024-03-11T19:58:33.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:32 smithi107 ceph-mon[31712]: pgmap v157: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 755 KiB/s wr, 138 op/s 2024-03-11T19:58:35.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:34 smithi122 ceph-mon[36397]: pgmap v158: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 698 KiB/s wr, 120 op/s 2024-03-11T19:58:35.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:34 smithi145 ceph-mon[36730]: pgmap v158: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 698 KiB/s wr, 120 op/s 2024-03-11T19:58:35.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:34 smithi107 ceph-mon[31712]: pgmap v158: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 698 KiB/s wr, 120 op/s 2024-03-11T19:58:37.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:36 smithi122 ceph-mon[36397]: pgmap v159: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 689 KiB/s wr, 110 op/s 2024-03-11T19:58:37.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:36 smithi145 ceph-mon[36730]: pgmap v159: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 689 KiB/s wr, 110 op/s 2024-03-11T19:58:37.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:36 smithi107 ceph-mon[31712]: pgmap v159: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 689 KiB/s wr, 110 op/s 2024-03-11T19:58:39.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:38 smithi122 ceph-mon[36397]: pgmap v160: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-11T19:58:39.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:38 smithi145 ceph-mon[36730]: pgmap v160: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-11T19:58:39.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:38 smithi107 ceph-mon[31712]: pgmap v160: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-03-11T19:58:41.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:40 smithi122 ceph-mon[36397]: pgmap v161: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 815 KiB/s wr, 128 op/s 2024-03-11T19:58:41.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:40 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:58:41.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:40 smithi145 ceph-mon[36730]: pgmap v161: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 815 KiB/s wr, 128 op/s 2024-03-11T19:58:41.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:40 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:58:41.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:40 smithi107 ceph-mon[31712]: pgmap v161: 129 pgs: 129 active+clean; 2.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 815 KiB/s wr, 128 op/s 2024-03-11T19:58:41.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:40 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:58:42.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:41 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:58:42.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:41 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:58:42.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:41 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:58:42.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:41 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:58:42.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:41 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:58:42.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:41 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:58:42.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:41 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:58:42.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:41 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:58:42.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:41 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:58:43.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:42 smithi122 ceph-mon[36397]: pgmap v162: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-11T19:58:43.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:42 smithi145 ceph-mon[36730]: pgmap v162: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-11T19:58:43.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:42 smithi107 ceph-mon[31712]: pgmap v162: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-11T19:58:45.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:44 smithi122 ceph-mon[36397]: pgmap v163: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-11T19:58:45.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:44 smithi145 ceph-mon[36730]: pgmap v163: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-11T19:58:45.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:44 smithi107 ceph-mon[31712]: pgmap v163: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 -- 172.21.15.107:0/3571875368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c102a50 msgr2=0x7fd88c10af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 --2- 172.21.15.107:0/3571875368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c102a50 0x7fd88c10af30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802bcd0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 -- 172.21.15.107:0/3571875368 shutdown_connections 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 --2- 172.21.15.107:0/3571875368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c10b550 0x7fd88c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 --2- 172.21.15.107:0/3571875368 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c102a50 0x7fd88c10af30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 --2- 172.21.15.107:0/3571875368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102130 0x7fd88c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 -- 172.21.15.107:0/3571875368 >> 172.21.15.107:0/3571875368 conn(0x7fd88c0fb990 msgr2=0x7fd88c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 -- 172.21.15.107:0/3571875368 shutdown_connections 2024-03-11T19:58:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.011+0000 7fd88be00640 1 -- 172.21.15.107:0/3571875368 wait complete. 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.012+0000 7fd88be00640 1 Processor -- start 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 -- start start 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 0x7fd88c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c10b550 0x7fd88c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19c4c0 con 0x7fd88c102130 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19c630 con 0x7fd88c102a50 2024-03-11T19:58:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c19c7f0 con 0x7fd88c10b550 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37390/0 (socket says 172.21.15.107:37390) 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88aa00640 1 -- 172.21.15.107:0/3698159383 learned_addr learned my addr 172.21.15.107:0/3698159383 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 0x7fd88c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.013+0000 7fd88b400640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c10b550 0x7fd88c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 -- 172.21.15.107:0/3698159383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c10b550 msgr2=0x7fd88c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c10b550 0x7fd88c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 -- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 msgr2=0x7fd88c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 -- 172.21.15.107:0/3698159383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009560 con 0x7fd88c102a50 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88aa00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88a000640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 0x7fd88c197b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd878036040 tx=0x7fd87802c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd883400640 1 -- 172.21.15.107:0/3698159383 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878004350 con 0x7fd88c102a50 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c19ca70 con 0x7fd88c102a50 2024-03-11T19:58:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd883400640 1 -- 172.21.15.107:0/3698159383 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8780044b0 con 0x7fd88c102a50 2024-03-11T19:58:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd883400640 1 -- 172.21.15.107:0/3698159383 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878003d30 con 0x7fd88c102a50 2024-03-11T19:58:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.014+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c1a4350 con 0x7fd88c102a50 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 Processor -- start 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 -- start start 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd883400640 1 -- 172.21.15.107:0/3698159383 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fd878004df0 con 0x7fd88c102a50 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf53cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf548960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf4f8620 con 0x5578bf546570 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf4aa8c0 con 0x5578bf53dd60 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.018+0000 7fd894814740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf4a7df0 con 0x5578bf53cff0 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf53cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38354/0 (socket says 172.21.15.107:38354) 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf548960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 -- 172.21.15.107:0/1963472330 learned_addr learned my addr 172.21.15.107:0/1963472330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 -- 172.21.15.107:0/1963472330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 msgr2=0x5578bf53cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf53cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 -- 172.21.15.107:0/1963472330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 msgr2=0x5578bf548960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf548960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 -- 172.21.15.107:0/1963472330 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578bf424b00 con 0x5578bf53cff0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd88aa00640 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0105c0 tx=0x7fd87c010a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=505d7e9292cce6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.019+0000 7fd877e00640 1 -- 172.21.15.107:0/1963472330 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00e850 con 0x5578bf53cff0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd883400640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 0x7fd8700a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd877e00640 1 -- 172.21.15.107:0/1963472330 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd87c015070 con 0x5578bf53cff0 2024-03-11T19:58:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd877e00640 1 -- 172.21.15.107:0/1963472330 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011df0 con 0x5578bf53cff0 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd883400640 1 -- 172.21.15.107:0/3698159383 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fd8780f0120 con 0x7fd88c102a50 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd88b400640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 0x7fd8700a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 -- 172.21.15.107:0/1963472330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 msgr2=0x5578bf53d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0105c0 tx=0x7fd87c010a80 comp rx=0 tx=0).stop 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 -- 172.21.15.107:0/1963472330 shutdown_connections 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf548960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf53d470 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 --2- 172.21.15.107:0/1963472330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf53cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.020+0000 7fd894814740 1 -- 172.21.15.107:0/1963472330 >> 172.21.15.107:0/1963472330 conn(0x5578bf5044b0 msgr2=0x5578bf537ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.021+0000 7fd894814740 1 -- 172.21.15.107:0/1963472330 shutdown_connections 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.021+0000 7fd88b400640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 0x7fd8700a4050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd884005fd0 tx=0x7fd884015040 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.021+0000 7fd894814740 1 -- 172.21.15.107:0/1963472330 wait complete. 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.021+0000 7fd894814740 1 -- start start 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.021+0000 7fd894814740 1 Processor -- start 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf543550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf543fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf4a35d0 con 0x5578bf546570 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf448b50 con 0x5578bf53cff0 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf4487f0 con 0x5578bf53dd60 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd894814740 10 client.0 _ll_register_callbacks cb 0x5578bf45c550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd88b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf543fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd88a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd88a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38366/0 (socket says 172.21.15.107:38366) 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd88a000640 1 -- 172.21.15.107:0/2665983590 learned_addr learned my addr 172.21.15.107:0/2665983590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.022+0000 7fd88aa00640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf543550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 -- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 msgr2=0x5578bf543550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf543550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 -- 172.21.15.107:0/2665983590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 msgr2=0x5578bf543fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf543fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 -- 172.21.15.107:0/2665983590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87c00be20 con 0x5578bf53dd60 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88aa00640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf543550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd88a000640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd878003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfa96a2fb8856dd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780bc380 con 0x5578bf53dd60 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fd8780dc040 con 0x5578bf53dd60 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578bf475cd0 con 0x5578bf53dd60 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8780bd120 con 0x5578bf53dd60 2024-03-11T19:58:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd894814740 10 client.25912 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.023+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578bf4a1c50 con 0x5578bf53dd60 2024-03-11T19:58:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8780d4070 con 0x5578bf53dd60 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd87803c070 con 0x5578bf53dd60 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 10 client.25912 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 10 client.25912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 20 client.25912 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5561e0 0x5578bf558600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 4 client.25912 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578bf499ce0 con 0x5578bf5561e0 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.024+0000 7fd88aa00640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5561e0 0x5578bf558600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.025+0000 7fd88aa00640 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5561e0 0x5578bf558600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9bee68fc150f7c server_cookie=b831ae04aa6ba693 in_seq=0 out_seq=0 2024-03-11T19:58:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.025+0000 7fd874c00640 10 client.25912 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.025+0000 7fd874c00640 1 -- 172.21.15.107:0/2665983590 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578bf499ce0 con 0x5578bf5561e0 2024-03-11T19:58:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.025+0000 7fd874c00640 10 client.25912 handle_command_reply: tid=0 2024-03-11T19:58:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.025+0000 7fd894814740 1 client.25912 shutdown 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 msgr2=0x5578bf543a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd878003f50 comp rx=0 tx=0).stop 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 shutdown_connections 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5561e0 0x5578bf558600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf543fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53dd60 0x5578bf543a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 --2- 172.21.15.107:0/2665983590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf543550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd874c00640 0 client.25912 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 >> 172.21.15.107:0/2665983590 conn(0x5578bf5044b0 msgr2=0x5578bf539a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 shutdown_connections 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 1 -- 172.21.15.107:0/2665983590 wait complete. 2024-03-11T19:58:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.026+0000 7fd894814740 20 client.25912 trim_cache size 0 max 16384 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.041+0000 7fd894814740 1 Processor -- start 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 -- start start 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 0x5578bf543df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf546570 0x5578bf54d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf1ebe60 con 0x5578bf53dd60 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf553050 con 0x5578bf53cff0 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd894814740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf537550 con 0x5578bf546570 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:51526/0 (socket says 172.21.15.107:51526) 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 -- 172.21.15.107:0/476323469 learned_addr learned my addr 172.21.15.107:0/476323469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88a000640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 0x5578bf543df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88b400640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf546570 0x5578bf54d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 -- 172.21.15.107:0/476323469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf546570 msgr2=0x5578bf54d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf546570 0x5578bf54d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 -- 172.21.15.107:0/476323469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 msgr2=0x5578bf543df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 0x5578bf543df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88aa00640 1 -- 172.21.15.107:0/476323469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578bf43e530 con 0x5578bf53cff0 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.042+0000 7fd88a000640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 0x5578bf543df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.043+0000 7fd88aa00640 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0105c0 tx=0x7fd87c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29400d7d336b8abe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.043+0000 7fd877400640 1 -- 172.21.15.107:0/476323469 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011590 con 0x5578bf53cff0 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.043+0000 7fd877400640 1 -- 172.21.15.107:0/476323469 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd87c015040 con 0x5578bf53cff0 2024-03-11T19:58:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.043+0000 7fd877400640 1 -- 172.21.15.107:0/476323469 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c0280f0 con 0x5578bf53cff0 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 -- 172.21.15.107:0/476323469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 msgr2=0x5578bf545c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0105c0 tx=0x7fd87c013040 comp rx=0 tx=0).stop 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 -- 172.21.15.107:0/476323469 shutdown_connections 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf546570 0x5578bf54d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf53dd60 0x5578bf543df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 --2- 172.21.15.107:0/476323469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53cff0 0x5578bf545c80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 -- 172.21.15.107:0/476323469 >> 172.21.15.107:0/476323469 conn(0x5578bf5044b0 msgr2=0x5578bf55cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 -- 172.21.15.107:0/476323469 shutdown_connections 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.044+0000 7fd894814740 1 -- 172.21.15.107:0/476323469 wait complete. 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.045+0000 7fd894814740 1 -- start start 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.045+0000 7fd894814740 1 Processor -- start 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf54d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf549850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf549f50 con 0x5578bf546570 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf54a0c0 con 0x5578bf53dd60 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bf54a280 con 0x5578bf53cff0 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd894814740 10 client.0 _ll_register_callbacks cb 0x7fd878005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd88b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf549850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd88aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:38380/0 (socket says 172.21.15.107:38380) 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd88aa00640 1 -- 172.21.15.107:0/4238195706 learned_addr learned my addr 172.21.15.107:0/4238195706 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.046+0000 7fd88a000640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf54d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 -- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 msgr2=0x5578bf54d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf54d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 -- 172.21.15.107:0/4238195706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 msgr2=0x5578bf549850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:58:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf549850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 -- 172.21.15.107:0/4238195706 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578bf424b00 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd88aa00640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005300 tx=0x7fd87c005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7840ee07fcbe76a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd874c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c0050a0 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578bf4c8d30 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.047+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mon.1 v2:172.21.15.145:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9517+0+0 (secure 0 0 0) 0x7fd87c005400 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87c0055c0 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 10 client.25918 fetch_fsmap learned FSMap version 28 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5578bf4642c0 con 0x5578bf53cff0 2024-03-11T19:58:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mon.1 v2:172.21.15.145:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578bf4c8d30 con 0x5578bf53cff0 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mon.1 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd87c028780 con 0x5578bf53cff0 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 10 client.25918 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 20 client.25918 populate_metadata read hostname 'smithi107' 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5ad590 0x5578bf5af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 4 client.25918 mds_command: new command op to 24590 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 --> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578bf1a3600 con 0x5578bf5ad590 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.048+0000 7fd88a000640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5ad590 0x5578bf5af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.049+0000 7fd88a000640 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5ad590 0x5578bf5af9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce28e4a484fad81 server_cookie=16a723ea13b17887 in_seq=0 out_seq=0 2024-03-11T19:58:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.049+0000 7fd874c00640 10 client.25918 ms_handle_connect on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.057+0000 7fd874c00640 1 -- 172.21.15.107:0/4238195706 <== mds.0 v2:172.21.15.122:6838/1167549353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578bf1a3600 con 0x5578bf5ad590 2024-03-11T19:58:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.057+0000 7fd874c00640 10 client.25918 handle_command_reply: tid=0 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.057+0000 7fd894814740 1 client.25918 shutdown 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 msgr2=0x5578bf54d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005300 tx=0x7fd87c005bf0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 shutdown_connections 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:6838/1167549353,v1:172.21.15.122:6839/1167549353] conn(0x5578bf5ad590 0x5578bf5af9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578bf53dd60 0x5578bf54d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578bf546570 0x5578bf549850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 --2- 172.21.15.107:0/4238195706 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5578bf53cff0 0x5578bf54d2d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd874c00640 0 client.25918 ms_handle_reset on v2:172.21.15.122:6838/1167549353 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 >> 172.21.15.107:0/4238195706 conn(0x5578bf5044b0 msgr2=0x5578bf541d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 shutdown_connections 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 1 -- 172.21.15.107:0/4238195706 wait complete. 2024-03-11T19:58:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.058+0000 7fd894814740 20 client.25918 trim_cache size 0 max 16384 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 msgr2=0x7fd8700a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 0x7fd8700a4050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd884005fd0 tx=0x7fd884015040 comp rx=0 tx=0).stop 2024-03-11T19:58:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 msgr2=0x7fd88c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 0x7fd88c197b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd878036040 tx=0x7fd87802c720 comp rx=0 tx=0).stop 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 shutdown_connections 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fd8700a1b90 0x7fd8700a4050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd88c102a50 0x7fd88c197b90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd88c10b550 0x7fd88c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 --2- 172.21.15.107:0/3698159383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c102130 0x7fd88c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 >> 172.21.15.107:0/3698159383 conn(0x7fd88c0fb990 msgr2=0x7fd88c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 shutdown_connections 2024-03-11T19:58:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-11T19:58:46.059+0000 7fd88be00640 1 -- 172.21.15.107:0/3698159383 wait complete. 2024-03-11T19:58:47.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:46 smithi122 ceph-mon[36397]: pgmap v164: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 900 KiB/s wr, 144 op/s 2024-03-11T19:58:47.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:46 smithi145 ceph-mon[36730]: pgmap v164: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 900 KiB/s wr, 144 op/s 2024-03-11T19:58:47.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:46 smithi107 ceph-mon[31712]: pgmap v164: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 900 KiB/s wr, 144 op/s 2024-03-11T19:58:49.007 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:48 smithi122 ceph-mon[36397]: pgmap v165: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-03-11T19:58:49.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:48 smithi145 ceph-mon[36730]: pgmap v165: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-03-11T19:58:49.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:48 smithi107 ceph-mon[31712]: pgmap v165: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-03-11T19:58:51.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:50 smithi122 ceph-mon[36397]: pgmap v166: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 917 KiB/s wr, 156 op/s 2024-03-11T19:58:51.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:50 smithi145 ceph-mon[36730]: pgmap v166: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 917 KiB/s wr, 156 op/s 2024-03-11T19:58:51.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:50 smithi107 ceph-mon[31712]: pgmap v166: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 917 KiB/s wr, 156 op/s 2024-03-11T19:58:53.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:52 smithi122 ceph-mon[36397]: pgmap v167: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-03-11T19:58:53.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:52 smithi145 ceph-mon[36730]: pgmap v167: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-03-11T19:58:53.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:52 smithi107 ceph-mon[31712]: pgmap v167: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-03-11T19:58:53.646 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-11T19:58:53.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-11T19:58:54.872 DEBUG:teuthology.parallel:result is None 2024-03-11T19:58:54.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T19:58:54.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:54 smithi107 ceph-mon[31712]: pgmap v168: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-03-11T19:58:54.967 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T19:58:54.967 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-11T19:58:54.977 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:58:55.002 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-11T19:58:55.013 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-11T19:58:55.013 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:58:55.013 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T19:58:55.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-11T19:58:55.014 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:54 smithi122 ceph-mon[36397]: pgmap v168: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-03-11T19:58:55.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:54 smithi145 ceph-mon[36730]: pgmap v168: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 201 op/s 2024-03-11T19:58:55.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-11T19:58:57.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:56 smithi122 ceph-mon[36397]: pgmap v169: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-03-11T19:58:57.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:56 smithi145 ceph-mon[36730]: pgmap v169: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-03-11T19:58:57.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:56 smithi107 ceph-mon[31712]: pgmap v169: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-03-11T19:58:59.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:58:58 smithi122 ceph-mon[36397]: pgmap v170: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-03-11T19:58:59.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:58:58 smithi145 ceph-mon[36730]: pgmap v170: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-03-11T19:58:59.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:58:58 smithi107 ceph-mon[31712]: pgmap v170: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-03-11T19:59:01.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:00 smithi122 ceph-mon[36397]: pgmap v171: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 885 KiB/s wr, 156 op/s 2024-03-11T19:59:01.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:00 smithi145 ceph-mon[36730]: pgmap v171: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 885 KiB/s wr, 156 op/s 2024-03-11T19:59:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:00 smithi107 ceph-mon[31712]: pgmap v171: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 885 KiB/s wr, 156 op/s 2024-03-11T19:59:03.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:02 smithi122 ceph-mon[36397]: pgmap v172: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 913 KiB/s wr, 162 op/s 2024-03-11T19:59:03.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:02 smithi145 ceph-mon[36730]: pgmap v172: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 913 KiB/s wr, 162 op/s 2024-03-11T19:59:03.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:02 smithi107 ceph-mon[31712]: pgmap v172: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 913 KiB/s wr, 162 op/s 2024-03-11T19:59:05.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:04 smithi122 ceph-mon[36397]: pgmap v173: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 707 KiB/s wr, 101 op/s 2024-03-11T19:59:05.032 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:04 smithi145 ceph-mon[36730]: pgmap v173: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 707 KiB/s wr, 101 op/s 2024-03-11T19:59:05.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:04 smithi107 ceph-mon[31712]: pgmap v173: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 707 KiB/s wr, 101 op/s 2024-03-11T19:59:07.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:06 smithi122 ceph-mon[36397]: pgmap v174: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 74 op/s 2024-03-11T19:59:07.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:06 smithi145 ceph-mon[36730]: pgmap v174: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 74 op/s 2024-03-11T19:59:07.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:06 smithi107 ceph-mon[31712]: pgmap v174: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 74 op/s 2024-03-11T19:59:09.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:08 smithi122 ceph-mon[36397]: pgmap v175: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 364 KiB/s wr, 78 op/s 2024-03-11T19:59:09.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:08 smithi145 ceph-mon[36730]: pgmap v175: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 364 KiB/s wr, 78 op/s 2024-03-11T19:59:09.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:08 smithi107 ceph-mon[31712]: pgmap v175: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 364 KiB/s wr, 78 op/s 2024-03-11T19:59:11.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:10 smithi122 ceph-mon[36397]: pgmap v176: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:59:11.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:10 smithi145 ceph-mon[36730]: pgmap v176: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:59:11.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:10 smithi107 ceph-mon[31712]: pgmap v176: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T19:59:13.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:12 smithi122 ceph-mon[36397]: pgmap v177: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-11T19:59:13.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:12 smithi145 ceph-mon[36730]: pgmap v177: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-11T19:59:13.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:12 smithi107 ceph-mon[31712]: pgmap v177: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-03-11T19:59:15.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:14 smithi122 ceph-mon[36397]: pgmap v178: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-03-11T19:59:15.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:14 smithi145 ceph-mon[36730]: pgmap v178: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-03-11T19:59:15.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:14 smithi107 ceph-mon[31712]: pgmap v178: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 21 op/s 2024-03-11T19:59:17.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:16 smithi122 ceph-mon[36397]: pgmap v179: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-03-11T19:59:17.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:16 smithi145 ceph-mon[36730]: pgmap v179: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-03-11T19:59:17.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:16 smithi107 ceph-mon[31712]: pgmap v179: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 6 op/s 2024-03-11T19:59:19.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:18 smithi122 ceph-mon[36397]: pgmap v180: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-03-11T19:59:19.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:18 smithi145 ceph-mon[36730]: pgmap v180: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-03-11T19:59:19.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:18 smithi107 ceph-mon[31712]: pgmap v180: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-03-11T19:59:21.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:20 smithi122 ceph-mon[36397]: pgmap v181: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:59:21.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:20 smithi145 ceph-mon[36730]: pgmap v181: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:59:21.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:20 smithi107 ceph-mon[31712]: pgmap v181: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:59:23.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:22 smithi122 ceph-mon[36397]: pgmap v182: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-11T19:59:23.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:22 smithi145 ceph-mon[36730]: pgmap v182: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-11T19:59:23.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:22 smithi107 ceph-mon[31712]: pgmap v182: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-11T19:59:25.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:24 smithi122 ceph-mon[36397]: pgmap v183: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:25.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:24 smithi145 ceph-mon[36730]: pgmap v183: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:25.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:24 smithi107 ceph-mon[31712]: pgmap v183: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:27.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:26 smithi122 ceph-mon[36397]: pgmap v184: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:27.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:26 smithi145 ceph-mon[36730]: pgmap v184: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:27.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:26 smithi107 ceph-mon[31712]: pgmap v184: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-11T19:59:29.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:28 smithi122 ceph-mon[36397]: pgmap v185: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:59:29.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:28 smithi145 ceph-mon[36730]: pgmap v185: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:59:29.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:28 smithi107 ceph-mon[31712]: pgmap v185: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-11T19:59:31.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:30 smithi145 ceph-mon[36730]: pgmap v186: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:31.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:30 smithi107 ceph-mon[31712]: pgmap v186: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:31.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:30 smithi122 ceph-mon[36397]: pgmap v186: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:33.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:32 smithi145 ceph-mon[36730]: pgmap v187: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:33.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:32 smithi107 ceph-mon[31712]: pgmap v187: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:33.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:32 smithi122 ceph-mon[36397]: pgmap v187: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:35.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:34 smithi145 ceph-mon[36730]: pgmap v188: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:35.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:34 smithi107 ceph-mon[31712]: pgmap v188: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:35.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:34 smithi122 ceph-mon[36397]: pgmap v188: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:37.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:36 smithi145 ceph-mon[36730]: pgmap v189: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:37.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:36 smithi107 ceph-mon[31712]: pgmap v189: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:37.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:36 smithi122 ceph-mon[36397]: pgmap v189: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:39.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:38 smithi145 ceph-mon[36730]: pgmap v190: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:39.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:38 smithi107 ceph-mon[31712]: pgmap v190: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:39.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:38 smithi122 ceph-mon[36397]: pgmap v190: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:41.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:40 smithi107 ceph-mon[31712]: pgmap v191: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:41.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:40 smithi122 ceph-mon[36397]: pgmap v191: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:41.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:40 smithi145 ceph-mon[36730]: pgmap v191: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:42.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:41 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:59:42.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:41 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:59:42.286 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:41 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:59:43.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:42 smithi122 ceph-mon[36397]: pgmap v192: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:43.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:42 smithi145 ceph-mon[36730]: pgmap v192: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:43.287 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:42 smithi107 ceph-mon[31712]: pgmap v192: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:44.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:43 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:59:44.255 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:43 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:59:44.256 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:43 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:59:44.256 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:43 smithi122 ceph-mon[36397]: pgmap v193: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:44.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:43 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:59:44.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:43 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:59:44.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:43 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:59:44.280 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:43 smithi145 ceph-mon[36730]: pgmap v193: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:44.287 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:43 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:59:44.287 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:43 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:59:44.287 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:43 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T19:59:44.287 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:43 smithi107 ceph-mon[31712]: pgmap v193: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:47.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:46 smithi122 ceph-mon[36397]: pgmap v194: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:47.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:46 smithi145 ceph-mon[36730]: pgmap v194: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:47.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:46 smithi107 ceph-mon[31712]: pgmap v194: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:49.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:48 smithi122 ceph-mon[36397]: pgmap v195: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:49.006 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:48 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:59:49.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:48 smithi145 ceph-mon[36730]: pgmap v195: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:49.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:48 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:59:49.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:48 smithi107 ceph-mon[31712]: pgmap v195: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:49.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:48 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:59:51.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:50 smithi122 ceph-mon[36397]: pgmap v196: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:51.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:50 smithi145 ceph-mon[36730]: pgmap v196: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:51.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:50 smithi107 ceph-mon[31712]: pgmap v196: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:53.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:52 smithi122 ceph-mon[36397]: pgmap v197: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:53.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:52 smithi145 ceph-mon[36730]: pgmap v197: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:53.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:52 smithi107 ceph-mon[31712]: pgmap v197: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T19:59:55.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:54 smithi122 ceph-mon[36397]: pgmap v198: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:55.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:54 smithi145 ceph-mon[36730]: pgmap v198: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:55.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:54 smithi107 ceph-mon[31712]: pgmap v198: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T19:59:57.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:56 smithi122 ceph-mon[36397]: pgmap v199: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:57.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:56 smithi145 ceph-mon[36730]: pgmap v199: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:57.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:56 smithi107 ceph-mon[31712]: pgmap v199: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:59:59.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 19:59:58 smithi122 ceph-mon[36397]: pgmap v200: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:59.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 19:59:58 smithi145 ceph-mon[36730]: pgmap v200: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:59:59.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 19:59:58 smithi107 ceph-mon[31712]: pgmap v200: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:01.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:00 smithi122 ceph-mon[36397]: pgmap v201: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:01.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:00 smithi122 ceph-mon[36397]: overall HEALTH_OK 2024-03-11T20:00:01.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:00 smithi145 ceph-mon[36730]: pgmap v201: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:01.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:00 smithi145 ceph-mon[36730]: overall HEALTH_OK 2024-03-11T20:00:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:00 smithi107 ceph-mon[31712]: pgmap v201: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:00 smithi107 ceph-mon[31712]: overall HEALTH_OK 2024-03-11T20:00:03.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:02 smithi122 ceph-mon[36397]: pgmap v202: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:00:03.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:02 smithi145 ceph-mon[36730]: pgmap v202: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:00:03.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:02 smithi107 ceph-mon[31712]: pgmap v202: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:00:05.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:04 smithi122 ceph-mon[36397]: pgmap v203: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:05.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:04 smithi145 ceph-mon[36730]: pgmap v203: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:05.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:04 smithi107 ceph-mon[31712]: pgmap v203: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:07.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:06 smithi122 ceph-mon[36397]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:07.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:06 smithi145 ceph-mon[36730]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:07.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:06 smithi107 ceph-mon[31712]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:09.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:08 smithi122 ceph-mon[36397]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:00:09.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:08 smithi145 ceph-mon[36730]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:00:09.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:08 smithi107 ceph-mon[31712]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:00:11.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:10 smithi122 ceph-mon[36397]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:11.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:10 smithi122 ceph-mon[36397]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T20:00:11.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:10 smithi145 ceph-mon[36730]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:11.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:10 smithi145 ceph-mon[36730]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T20:00:11.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:10 smithi107 ceph-mon[31712]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:00:11.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:10 smithi107 ceph-mon[31712]: osdmap e98: 12 total, 12 up, 12 in 2024-03-11T20:00:13.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:12 smithi122 ceph-mon[36397]: pgmap v208: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:00:13.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:12 smithi145 ceph-mon[36730]: pgmap v208: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:00:13.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:12 smithi107 ceph-mon[31712]: pgmap v208: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:00:15.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:14 smithi122 ceph-mon[36397]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:15.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:14 smithi145 ceph-mon[36730]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:15.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:14 smithi107 ceph-mon[31712]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:17.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:16 smithi122 ceph-mon[36397]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:17.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:16 smithi145 ceph-mon[36730]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:17.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:16 smithi107 ceph-mon[31712]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:19.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:18 smithi122 ceph-mon[36397]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:19.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:18 smithi122 ceph-mon[36397]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T20:00:19.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:18 smithi145 ceph-mon[36730]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:19.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:18 smithi145 ceph-mon[36730]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T20:00:19.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:18 smithi107 ceph-mon[31712]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:00:19.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:18 smithi107 ceph-mon[31712]: osdmap e99: 12 total, 12 up, 12 in 2024-03-11T20:00:21.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:20 smithi122 ceph-mon[36397]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-03-11T20:00:21.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:20 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:00:21.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:20 smithi145 ceph-mon[36730]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-03-11T20:00:21.031 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:20 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:00:21.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:20 smithi107 ceph-mon[31712]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-03-11T20:00:21.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:20 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:00:23.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:22 smithi122 ceph-mon[36397]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:23.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:22 smithi145 ceph-mon[36730]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:23.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:22 smithi107 ceph-mon[31712]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:25.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:24 smithi122 ceph-mon[36397]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:25.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:24 smithi145 ceph-mon[36730]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:25.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:24 smithi107 ceph-mon[31712]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:27.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:26 smithi122 ceph-mon[36397]: pgmap v216: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:27.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:26 smithi145 ceph-mon[36730]: pgmap v216: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:27.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:26 smithi107 ceph-mon[31712]: pgmap v216: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:00:29.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:28 smithi122 ceph-mon[36397]: pgmap v217: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:00:29.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:28 smithi145 ceph-mon[36730]: pgmap v217: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:00:29.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:28 smithi107 ceph-mon[31712]: pgmap v217: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:00:31.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:30 smithi122 ceph-mon[36397]: pgmap v218: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:00:31.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:30 smithi145 ceph-mon[36730]: pgmap v218: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:00:31.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:30 smithi107 ceph-mon[31712]: pgmap v218: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:00:33.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:32 smithi122 ceph-mon[36397]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:33.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:32 smithi145 ceph-mon[36730]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:33.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:32 smithi107 ceph-mon[31712]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:35.005 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:34 smithi122 ceph-mon[36397]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:35.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:34 smithi145 ceph-mon[36730]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:35.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:34 smithi107 ceph-mon[31712]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:37.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:36 smithi122 ceph-mon[36397]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:37.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:36 smithi145 ceph-mon[36730]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:37.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:36 smithi107 ceph-mon[31712]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:39.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:38 smithi122 ceph-mon[36397]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:39.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:38 smithi145 ceph-mon[36730]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:39.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:38 smithi107 ceph-mon[31712]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:41.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:40 smithi122 ceph-mon[36397]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:41.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:40 smithi145 ceph-mon[36730]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:41.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:40 smithi107 ceph-mon[31712]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:43.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:42 smithi122 ceph-mon[36397]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:43.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:42 smithi145 ceph-mon[36730]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:43.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:42 smithi107 ceph-mon[31712]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:00:44.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:43 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:00:44.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:43 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:00:44.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:43 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:00:45.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:44 smithi122 ceph-mon[36397]: pgmap v225: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:45.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:44 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:00:45.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:44 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:00:45.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:44 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:00:45.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:44 smithi145 ceph-mon[36730]: pgmap v225: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:45.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:44 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:00:45.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:44 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:00:45.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:44 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:00:45.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:44 smithi107 ceph-mon[31712]: pgmap v225: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:00:45.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:44 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:00:45.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:44 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:00:45.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:44 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:00:47.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:46 smithi122 ceph-mon[36397]: pgmap v226: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:47.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:46 smithi145 ceph-mon[36730]: pgmap v226: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:47.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:46 smithi107 ceph-mon[31712]: pgmap v226: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:00:49.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:48 smithi122 ceph-mon[36397]: pgmap v227: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:49.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:48 smithi145 ceph-mon[36730]: pgmap v227: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:49.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:48 smithi107 ceph-mon[31712]: pgmap v227: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:51.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:50 smithi122 ceph-mon[36397]: pgmap v228: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:51.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:50 smithi145 ceph-mon[36730]: pgmap v228: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:51.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:50 smithi107 ceph-mon[31712]: pgmap v228: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:53.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:52 smithi122 ceph-mon[36397]: pgmap v229: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T20:00:53.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:52 smithi145 ceph-mon[36730]: pgmap v229: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T20:00:53.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:52 smithi107 ceph-mon[31712]: pgmap v229: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-11T20:00:55.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:54 smithi122 ceph-mon[36397]: pgmap v230: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-11T20:00:55.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:54 smithi145 ceph-mon[36730]: pgmap v230: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-11T20:00:55.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:54 smithi107 ceph-mon[31712]: pgmap v230: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-11T20:00:57.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:56 smithi122 ceph-mon[36397]: pgmap v231: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:57.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:56 smithi145 ceph-mon[36730]: pgmap v231: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:57.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:56 smithi107 ceph-mon[31712]: pgmap v231: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-11T20:00:59.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:58 smithi122 ceph-mon[36397]: pgmap v232: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:59.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:58 smithi145 ceph-mon[36730]: pgmap v232: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:00:59.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:58 smithi107 ceph-mon[31712]: pgmap v232: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:00.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:00:59 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:00.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:00:59 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:00.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:00:59 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:01.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:00 smithi145 ceph-mon[36730]: pgmap v233: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:01.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:00 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:00 smithi107 ceph-mon[31712]: pgmap v233: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:00 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:01.150 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:00 smithi122 ceph-mon[36397]: pgmap v233: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:01.150 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:00 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:03.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:02 smithi107 ceph-mon[31712]: pgmap v234: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:01:03.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:02 smithi122 ceph-mon[36397]: pgmap v234: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:01:03.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:02 smithi145 ceph-mon[36730]: pgmap v234: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:01:05.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:04 smithi107 ceph-mon[31712]: pgmap v235: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:05.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:04 smithi122 ceph-mon[36397]: pgmap v235: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:05.279 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:04 smithi145 ceph-mon[36730]: pgmap v235: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:07.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:06 smithi122 ceph-mon[36397]: pgmap v236: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:07.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:06 smithi145 ceph-mon[36730]: pgmap v236: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:07.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:06 smithi107 ceph-mon[31712]: pgmap v236: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:09.081 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:08 smithi145 ceph-mon[36730]: pgmap v237: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:01:09.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:08 smithi122 ceph-mon[36397]: pgmap v237: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:01:09.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:08 smithi107 ceph-mon[31712]: pgmap v237: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-11T20:01:11.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:10 smithi107 ceph-mon[31712]: pgmap v238: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:11.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:10 smithi107 ceph-mon[31712]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T20:01:11.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:10 smithi122 ceph-mon[36397]: pgmap v238: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:11.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:10 smithi122 ceph-mon[36397]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T20:01:11.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:10 smithi145 ceph-mon[36730]: pgmap v238: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-11T20:01:11.279 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:10 smithi145 ceph-mon[36730]: osdmap e100: 12 total, 12 up, 12 in 2024-03-11T20:01:12.254 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:11 smithi122 ceph-mon[36397]: pgmap v240: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:01:12.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:11 smithi145 ceph-mon[36730]: pgmap v240: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:01:12.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:11 smithi107 ceph-mon[31712]: pgmap v240: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:01:15.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:14 smithi122 ceph-mon[36397]: pgmap v241: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:15.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:14 smithi145 ceph-mon[36730]: pgmap v241: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:15.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:14 smithi107 ceph-mon[31712]: pgmap v241: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:17.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:16 smithi122 ceph-mon[36397]: pgmap v242: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:17.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:16 smithi145 ceph-mon[36730]: pgmap v242: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:17.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:16 smithi107 ceph-mon[31712]: pgmap v242: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:19.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:18 smithi122 ceph-mon[36397]: pgmap v243: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:19.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:18 smithi122 ceph-mon[36397]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T20:01:19.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:18 smithi145 ceph-mon[36730]: pgmap v243: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:19.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:18 smithi145 ceph-mon[36730]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T20:01:19.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:18 smithi107 ceph-mon[31712]: pgmap v243: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:01:19.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:18 smithi107 ceph-mon[31712]: osdmap e101: 12 total, 12 up, 12 in 2024-03-11T20:01:21.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:20 smithi122 ceph-mon[36397]: pgmap v245: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-11T20:01:21.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:20 smithi145 ceph-mon[36730]: pgmap v245: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-11T20:01:21.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:20 smithi107 ceph-mon[31712]: pgmap v245: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-03-11T20:01:23.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:22 smithi122 ceph-mon[36397]: pgmap v246: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:23.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:22 smithi145 ceph-mon[36730]: pgmap v246: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:23.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:22 smithi107 ceph-mon[31712]: pgmap v246: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:25.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:24 smithi122 ceph-mon[36397]: pgmap v247: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:25.029 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:24 smithi145 ceph-mon[36730]: pgmap v247: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:25.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:24 smithi107 ceph-mon[31712]: pgmap v247: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:27.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:26 smithi122 ceph-mon[36397]: pgmap v248: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:27.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:26 smithi145 ceph-mon[36730]: pgmap v248: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:27.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:26 smithi107 ceph-mon[31712]: pgmap v248: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:01:29.004 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:28 smithi122 ceph-mon[36397]: pgmap v249: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:01:29.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:28 smithi145 ceph-mon[36730]: pgmap v249: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:01:29.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:28 smithi107 ceph-mon[31712]: pgmap v249: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:01:31.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:30 smithi122 ceph-mon[36397]: pgmap v250: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:01:31.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:30 smithi145 ceph-mon[36730]: pgmap v250: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:01:31.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:30 smithi107 ceph-mon[31712]: pgmap v250: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T20:01:33.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:32 smithi122 ceph-mon[36397]: pgmap v251: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:33.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:32 smithi145 ceph-mon[36730]: pgmap v251: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:33.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:32 smithi107 ceph-mon[31712]: pgmap v251: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:35.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:34 smithi122 ceph-mon[36397]: pgmap v252: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:35.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:34 smithi145 ceph-mon[36730]: pgmap v252: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:35.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:34 smithi107 ceph-mon[31712]: pgmap v252: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:37.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:36 smithi122 ceph-mon[36397]: pgmap v253: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:37.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:36 smithi145 ceph-mon[36730]: pgmap v253: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:37.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:36 smithi107 ceph-mon[31712]: pgmap v253: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:38.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:37 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:38.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:37 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:38.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:37 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:39.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:38 smithi122 ceph-mon[36397]: pgmap v254: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:39.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:38 smithi145 ceph-mon[36730]: pgmap v254: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:39.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:38 smithi107 ceph-mon[31712]: pgmap v254: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:40.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:39 smithi122 ceph-mon[36397]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:40.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:39 smithi145 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:40.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:39 smithi107 ceph-mon[31712]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T20:01:41.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:40 smithi122 ceph-mon[36397]: pgmap v255: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:41.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:40 smithi145 ceph-mon[36730]: pgmap v255: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:41.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:40 smithi107 ceph-mon[31712]: pgmap v255: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:43.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:42 smithi145 ceph-mon[36730]: pgmap v256: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:43.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:42 smithi107 ceph-mon[31712]: pgmap v256: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:43.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:42 smithi122 ceph-mon[36397]: pgmap v256: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:45.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:44 smithi145 ceph-mon[36730]: pgmap v257: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:45.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:44 smithi107 ceph-mon[31712]: pgmap v257: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:45.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:44 smithi122 ceph-mon[36397]: pgmap v257: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:46.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:45 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:01:46.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:45 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:01:46.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:45 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:01:47.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:46 smithi145 ceph-mon[36730]: pgmap v258: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:47.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:46 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:01:47.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:46 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:01:47.028 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:46 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:01:47.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:46 smithi107 ceph-mon[31712]: pgmap v258: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:47.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:46 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:01:47.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:46 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:01:47.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:46 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:01:47.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:46 smithi122 ceph-mon[36397]: pgmap v258: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:47.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:46 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:01:47.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:46 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:01:47.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:46 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:01:49.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:48 smithi107 ceph-mon[31712]: pgmap v259: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:49.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:48 smithi122 ceph-mon[36397]: pgmap v259: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:49.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:48 smithi145 ceph-mon[36730]: pgmap v259: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:51.055 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:50 smithi122 ceph-mon[36397]: pgmap v260: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:51.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:50 smithi145 ceph-mon[36730]: pgmap v260: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:51.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:50 smithi107 ceph-mon[31712]: pgmap v260: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:53.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:52 smithi122 ceph-mon[36397]: pgmap v261: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:53.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:52 smithi145 ceph-mon[36730]: pgmap v261: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:53.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:52 smithi107 ceph-mon[31712]: pgmap v261: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:01:55.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:54 smithi122 ceph-mon[36397]: pgmap v262: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:55.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:54 smithi145 ceph-mon[36730]: pgmap v262: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:55.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:54 smithi107 ceph-mon[31712]: pgmap v262: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:01:57.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:56 smithi122 ceph-mon[36397]: pgmap v263: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:57.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:56 smithi145 ceph-mon[36730]: pgmap v263: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:57.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:56 smithi107 ceph-mon[31712]: pgmap v263: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:01:59.088 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:01:58 smithi107 ceph-mon[31712]: pgmap v264: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:59.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:01:58 smithi122 ceph-mon[36397]: pgmap v264: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:01:59.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:01:58 smithi145 ceph-mon[36730]: pgmap v264: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:01.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:00 smithi122 ceph-mon[36397]: pgmap v265: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:01.278 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:00 smithi145 ceph-mon[36730]: pgmap v265: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:01.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:00 smithi107 ceph-mon[31712]: pgmap v265: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:02.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:01 smithi122 ceph-mon[36397]: pgmap v266: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:02:02.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:01 smithi145 ceph-mon[36730]: pgmap v266: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:02:02.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:01 smithi107 ceph-mon[31712]: pgmap v266: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:02:05.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:04 smithi122 ceph-mon[36397]: pgmap v267: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:05.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:04 smithi145 ceph-mon[36730]: pgmap v267: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:05.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:04 smithi107 ceph-mon[31712]: pgmap v267: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:07.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:06 smithi122 ceph-mon[36397]: pgmap v268: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:07.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:06 smithi145 ceph-mon[36730]: pgmap v268: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:07.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:06 smithi107 ceph-mon[31712]: pgmap v268: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:09.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:08 smithi122 ceph-mon[36397]: pgmap v269: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:02:09.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:08 smithi145 ceph-mon[36730]: pgmap v269: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:02:09.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:08 smithi107 ceph-mon[31712]: pgmap v269: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:02:11.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:10 smithi122 ceph-mon[36397]: pgmap v270: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:11.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:10 smithi122 ceph-mon[36397]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T20:02:11.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:10 smithi145 ceph-mon[36730]: pgmap v270: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:11.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:10 smithi145 ceph-mon[36730]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T20:02:11.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:10 smithi107 ceph-mon[31712]: pgmap v270: 129 pgs: 129 active+clean; 2.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:02:11.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:10 smithi107 ceph-mon[31712]: osdmap e102: 12 total, 12 up, 12 in 2024-03-11T20:02:13.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:12 smithi122 ceph-mon[36397]: pgmap v272: 129 pgs: 16 active+clean+snaptrim, 70 active+clean, 43 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:02:13.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:12 smithi145 ceph-mon[36730]: pgmap v272: 129 pgs: 16 active+clean+snaptrim, 70 active+clean, 43 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:02:13.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:12 smithi107 ceph-mon[31712]: pgmap v272: 129 pgs: 16 active+clean+snaptrim, 70 active+clean, 43 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:02:15.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:14 smithi122 ceph-mon[36397]: pgmap v273: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:15.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:14 smithi145 ceph-mon[36730]: pgmap v273: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:15.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:14 smithi107 ceph-mon[31712]: pgmap v273: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:17.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:16 smithi122 ceph-mon[36397]: pgmap v274: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:17.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:16 smithi145 ceph-mon[36730]: pgmap v274: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:17.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:16 smithi107 ceph-mon[31712]: pgmap v274: 129 pgs: 20 active+clean+snaptrim, 55 active+clean, 54 active+clean+snaptrim_wait; 2.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:19.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:18 smithi122 ceph-mon[36397]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:19.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:18 smithi145 ceph-mon[36730]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:19.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:18 smithi107 ceph-mon[31712]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:21.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:20 smithi122 ceph-mon[36397]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:21.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:20 smithi145 ceph-mon[36730]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:21.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:20 smithi107 ceph-mon[31712]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:02:23.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:22 smithi122 ceph-mon[36397]: pgmap v277: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 171 B/s wr, 7 op/s 2024-03-11T20:02:23.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:22 smithi122 ceph-mon[36397]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T20:02:23.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:22 smithi145 ceph-mon[36730]: pgmap v277: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 171 B/s wr, 7 op/s 2024-03-11T20:02:23.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:22 smithi145 ceph-mon[36730]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T20:02:23.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:22 smithi107 ceph-mon[31712]: pgmap v277: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 171 B/s wr, 7 op/s 2024-03-11T20:02:23.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:22 smithi107 ceph-mon[31712]: osdmap e103: 12 total, 12 up, 12 in 2024-03-11T20:02:25.030 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:24 smithi145 ceph-mon[36730]: pgmap v279: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:25.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:24 smithi107 ceph-mon[31712]: pgmap v279: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:25.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:24 smithi122 ceph-mon[36397]: pgmap v279: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:27.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:26 smithi145 ceph-mon[36730]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:27.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:26 smithi107 ceph-mon[31712]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:27.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:26 smithi122 ceph-mon[36397]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:02:29.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:28 smithi145 ceph-mon[36730]: pgmap v281: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:29.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:28 smithi107 ceph-mon[31712]: pgmap v281: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:29.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:28 smithi122 ceph-mon[36397]: pgmap v281: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:31.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:30 smithi145 ceph-mon[36730]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:31.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:30 smithi107 ceph-mon[31712]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:31.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:30 smithi122 ceph-mon[36397]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:33.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:32 smithi145 ceph-mon[36730]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:33.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:32 smithi107 ceph-mon[31712]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:33.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:32 smithi122 ceph-mon[36397]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:02:35.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:34 smithi107 ceph-mon[31712]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T20:02:35.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:34 smithi122 ceph-mon[36397]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T20:02:35.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:34 smithi145 ceph-mon[36730]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-11T20:02:37.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:36 smithi122 ceph-mon[36397]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:37.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:36 smithi145 ceph-mon[36730]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:37.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:36 smithi107 ceph-mon[31712]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:39.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:38 smithi122 ceph-mon[36397]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:39.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:38 smithi145 ceph-mon[36730]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:39.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:38 smithi107 ceph-mon[31712]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:41.079 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:40 smithi145 ceph-mon[36730]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:41.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:40 smithi122 ceph-mon[36397]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:41.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:40 smithi107 ceph-mon[31712]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:43.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:42 smithi122 ceph-mon[36397]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:43.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:42 smithi145 ceph-mon[36730]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:43.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:42 smithi107 ceph-mon[31712]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:45.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:44 smithi122 ceph-mon[36397]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:45.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:44 smithi145 ceph-mon[36730]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:45.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:44 smithi107 ceph-mon[31712]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:47.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:46 smithi122 ceph-mon[36397]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:47.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:46 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:02:47.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:46 smithi145 ceph-mon[36730]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:47.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:46 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:02:47.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:46 smithi107 ceph-mon[31712]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:47.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:46 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:02:48.502 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:48 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.502 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:48 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.502 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:48 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.503 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:48 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.503 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:48 smithi122 ceph-mon[36397]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:48.527 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:48 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.527 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:48 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.527 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:48 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.527 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:48 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.527 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:48 smithi145 ceph-mon[36730]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:48.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:48 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:48 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:48 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:48 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:48.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:48 smithi107 ceph-mon[31712]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:49.752 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:49 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:02:49.752 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:49 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:02:49.752 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:49 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:49.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:49 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:02:49.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:49 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:02:49.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:49 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:49.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:49 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:02:49.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:49 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:02:49.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:49 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:02:50.752 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:50 smithi122 ceph-mon[36397]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:50.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:50 smithi145 ceph-mon[36730]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:50.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:50 smithi107 ceph-mon[31712]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:53.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:52 smithi122 ceph-mon[36397]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:53.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:52 smithi145 ceph-mon[36730]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:53.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:52 smithi107 ceph-mon[31712]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:02:55.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:54 smithi122 ceph-mon[36397]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:55.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:54 smithi145 ceph-mon[36730]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:55.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:54 smithi107 ceph-mon[31712]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:02:57.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:56 smithi122 ceph-mon[36397]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:57.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:56 smithi145 ceph-mon[36730]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:57.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:56 smithi107 ceph-mon[31712]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:02:59.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:02:58 smithi122 ceph-mon[36397]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:59.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:02:58 smithi145 ceph-mon[36730]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:02:59.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:02:58 smithi107 ceph-mon[31712]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:01.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:00 smithi122 ceph-mon[36397]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:01.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:00 smithi145 ceph-mon[36730]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:00 smithi107 ceph-mon[31712]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:03.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:02 smithi122 ceph-mon[36397]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:03:03.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:02 smithi145 ceph-mon[36730]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:03:03.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:02 smithi107 ceph-mon[31712]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:03:05.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:04 smithi145 ceph-mon[36730]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:05.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:04 smithi107 ceph-mon[31712]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:05.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:04 smithi122 ceph-mon[36397]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:07.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:06 smithi145 ceph-mon[36730]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:07.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:06 smithi107 ceph-mon[31712]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:07.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:06 smithi122 ceph-mon[36397]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:09.027 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:08 smithi145 ceph-mon[36730]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:03:09.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:08 smithi107 ceph-mon[31712]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:03:09.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:08 smithi122 ceph-mon[36397]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:03:11.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:10 smithi107 ceph-mon[31712]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:11.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:10 smithi107 ceph-mon[31712]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T20:03:11.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:10 smithi122 ceph-mon[36397]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:11.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:10 smithi122 ceph-mon[36397]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T20:03:11.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:10 smithi145 ceph-mon[36730]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:03:11.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:10 smithi145 ceph-mon[36730]: osdmap e104: 12 total, 12 up, 12 in 2024-03-11T20:03:13.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:12 smithi107 ceph-mon[31712]: pgmap v304: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:03:13.078 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:12 smithi145 ceph-mon[36730]: pgmap v304: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:03:13.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:12 smithi122 ceph-mon[36397]: pgmap v304: 129 pgs: 16 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 70 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:03:15.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:14 smithi122 ceph-mon[36397]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:15.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:14 smithi145 ceph-mon[36730]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:15.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:14 smithi107 ceph-mon[31712]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:17.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:16 smithi122 ceph-mon[36397]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:17.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:16 smithi145 ceph-mon[36730]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:17.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:16 smithi107 ceph-mon[31712]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:18 smithi122 ceph-mon[36397]: pgmap v307: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:18 smithi145 ceph-mon[36730]: pgmap v307: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:19.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:18 smithi107 ceph-mon[31712]: pgmap v307: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:21.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:20 smithi122 ceph-mon[36397]: pgmap v308: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:21.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:20 smithi145 ceph-mon[36730]: pgmap v308: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:20 smithi107 ceph-mon[31712]: pgmap v308: 129 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 107 active+clean; 471 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:03:23.054 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:22 smithi122 ceph-mon[36397]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 7 op/s 2024-03-11T20:03:23.054 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:22 smithi122 ceph-mon[36397]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T20:03:23.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:22 smithi145 ceph-mon[36730]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 7 op/s 2024-03-11T20:03:23.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:22 smithi145 ceph-mon[36730]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T20:03:23.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:22 smithi107 ceph-mon[31712]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 172 B/s wr, 7 op/s 2024-03-11T20:03:23.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:22 smithi107 ceph-mon[31712]: osdmap e105: 12 total, 12 up, 12 in 2024-03-11T20:03:25.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:24 smithi122 ceph-mon[36397]: pgmap v311: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:25.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:24 smithi145 ceph-mon[36730]: pgmap v311: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:25.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:24 smithi107 ceph-mon[31712]: pgmap v311: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:27.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:26 smithi122 ceph-mon[36397]: pgmap v312: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:27.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:26 smithi145 ceph-mon[36730]: pgmap v312: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:27.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:26 smithi107 ceph-mon[31712]: pgmap v312: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-11T20:03:28.242 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-11T20:03:28.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 -- 172.21.15.107:0/1992355659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81086a0 msgr2=0x7f11e8110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 --2- 172.21.15.107:0/1992355659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81086a0 0x7f11e8110270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009cd0 tx=0x7f11e002c0e0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 -- 172.21.15.107:0/1992355659 shutdown_connections 2024-03-11T20:03:28.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 --2- 172.21.15.107:0/1992355659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81086a0 0x7f11e8110270 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 --2- 172.21.15.107:0/1992355659 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8103f80 0x7f11e8107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 --2- 172.21.15.107:0/1992355659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81035d0 0x7f11e81039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 -- 172.21.15.107:0/1992355659 >> 172.21.15.107:0/1992355659 conn(0x7f11e80fee30 msgr2=0x7f11e8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:28.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 -- 172.21.15.107:0/1992355659 shutdown_connections 2024-03-11T20:03:28.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.462+0000 7f11eda00640 1 -- 172.21.15.107:0/1992355659 wait complete. 2024-03-11T20:03:28.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.463+0000 7f11eda00640 1 Processor -- start 2024-03-11T20:03:28.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.463+0000 7f11eda00640 1 -- start start 2024-03-11T20:03:28.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81086a0 0x7f11e819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8199710 0x7f11e8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a1e0 con 0x7f11e81035d0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a350 con 0x7f11e81086a0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11eda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a510 con 0x7f11e8199710 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37780/0 (socket says 172.21.15.107:37780) 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11e6a00640 1 -- 172.21.15.107:0/813846451 learned_addr learned my addr 172.21.15.107:0/813846451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11e6000640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81086a0 0x7f11e819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.464+0000 7f11e7400640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8199710 0x7f11e8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 -- 172.21.15.107:0/813846451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8199710 msgr2=0x7f11e8199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8199710 0x7f11e8199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 -- 172.21.15.107:0/813846451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81086a0 msgr2=0x7f11e819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81086a0 0x7f11e819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 -- 172.21.15.107:0/813846451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0009900 con 0x7f11e81035d0 2024-03-11T20:03:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11e6a00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8009700 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e81a64b0 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11d800cd80 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8018920 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.465+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e81a6990 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.467+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e8104aa0 con 0x7f11e81035d0 2024-03-11T20:03:28.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.468+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f11d8018a80 con 0x7f11e81035d0 2024-03-11T20:03:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.470+0000 7f11df400640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 0x7f11cc0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.470+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f11d8014070 con 0x7f11e81035d0 2024-03-11T20:03:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.470+0000 7f11e6000640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 0x7f11cc0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.471+0000 7f11e6000640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 0x7f11cc0a4100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11e819af60 tx=0x7f11d400b390 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:28.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.474+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11d8052d20 con 0x7f11e81035d0 2024-03-11T20:03:28.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.701+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e8103ff0 con 0x7f11cc0a1c40 2024-03-11T20:03:28.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.718+0000 7f11df400640 1 -- 172.21.15.107:0/813846451 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f11e8103ff0 con 0x7f11cc0a1c40 2024-03-11T20:03:28.712 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-11T00:00:00", "created": "2024-03-11T19:53:26", "first": "2024-03-11T19:54:00", "last": "2024-03-11T20:03:00", "last_pruned": "2024-03-11T20:03:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-11T20:03:28.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.720+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 msgr2=0x7f11cc0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.720+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 0x7f11cc0a4100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11e819af60 tx=0x7f11d400b390 comp rx=0 tx=0).stop 2024-03-11T20:03:28.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.720+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 msgr2=0x7f11e819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.720+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 shutdown_connections 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f11cc0a1c40 0x7f11cc0a4100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11e81086a0 0x7f11e819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8199710 0x7f11e8199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 --2- 172.21.15.107:0/813846451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11e81035d0 0x7f11e819f650 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 >> 172.21.15.107:0/813846451 conn(0x7f11e80fee30 msgr2=0x7f11e80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 shutdown_connections 2024-03-11T20:03:28.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.721+0000 7f11eda00640 1 -- 172.21.15.107:0/813846451 wait complete. 2024-03-11T20:03:28.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 -- 172.21.15.107:0/2642987121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 msgr2=0x7f63bc1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 --2- 172.21.15.107:0/2642987121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc1039b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009930 tx=0x7f63b002bc20 comp rx=0 tx=0).stop 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 -- 172.21.15.107:0/2642987121 shutdown_connections 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 --2- 172.21.15.107:0/2642987121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc103f80 0x7f63bc107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 --2- 172.21.15.107:0/2642987121 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 0x7f63bc110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 --2- 172.21.15.107:0/2642987121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc1039b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 -- 172.21.15.107:0/2642987121 >> 172.21.15.107:0/2642987121 conn(0x7f63bc0fee30 msgr2=0x7f63bc101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:28.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 -- 172.21.15.107:0/2642987121 shutdown_connections 2024-03-11T20:03:28.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.942+0000 7f63c4400640 1 -- 172.21.15.107:0/2642987121 wait complete. 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.943+0000 7f63c4400640 1 Processor -- start 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.943+0000 7f63c4400640 1 -- start start 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 0x7f63bc19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 0x7f63bc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19a2b0 con 0x7f63bc1035d0 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19a420 con 0x7f63bc199800 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c4400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19a590 con 0x7f63bc1086a0 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 0x7f63bc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37804/0 (socket says 172.21.15.107:37804) 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 -- 172.21.15.107:0/3247792627 learned_addr learned my addr 172.21.15.107:0/3247792627 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1000640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 0x7f63bc19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 -- 172.21.15.107:0/3247792627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 msgr2=0x7f63bc19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 0x7f63bc19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 -- 172.21.15.107:0/3247792627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 msgr2=0x7f63bc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.944+0000 7f63c1a00640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 0x7f63bc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63c1a00640 1 -- 172.21.15.107:0/3247792627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009560 con 0x7f63bc1035d0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63c2400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 0x7f63bc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63c1a00640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009230 tx=0x7f63b002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0004350 con 0x7f63bc1035d0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63b00044b0 con 0x7f63bc1035d0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0003d00 con 0x7f63bc1035d0 2024-03-11T20:03:28.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc1a6490 con 0x7f63bc1035d0 2024-03-11T20:03:28.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.945+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc1a6970 con 0x7f63bc1035d0 2024-03-11T20:03:28.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.947+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc104aa0 con 0x7f63bc1035d0 2024-03-11T20:03:28.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.948+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f63b0004df0 con 0x7f63bc1035d0 2024-03-11T20:03:28.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.950+0000 7f63ba000640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 0x7f63a40a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:28.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.950+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f63b00efb40 con 0x7f63bc1035d0 2024-03-11T20:03:28.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.950+0000 7f63c1000640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 0x7f63a40a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:28.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.951+0000 7f63c1000640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 0x7f63a40a4100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63bc109bf0 tx=0x7f63a8005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:28.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:28.954+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63b00b86b0 con 0x7f63bc1035d0 2024-03-11T20:03:29.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:28 smithi122 ceph-mon[36397]: pgmap v313: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:03:29.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:28 smithi122 ceph-mon[36397]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T20:03:29.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:28 smithi145 ceph-mon[36730]: pgmap v313: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:03:29.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:28 smithi145 ceph-mon[36730]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T20:03:29.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:28 smithi107 ceph-mon[31712]: pgmap v313: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-11T20:03:29.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:28 smithi107 ceph-mon[31712]: osdmap e106: 12 total, 12 up, 12 in 2024-03-11T20:03:29.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.174+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 --> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc10b940 con 0x7f63a40a1c40 2024-03-11T20:03:29.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.190+0000 7f63ba000640 1 -- 172.21.15.107:0/3247792627 <== mgr.15324 v2:172.21.15.107:6800/4104906604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f63bc10b940 con 0x7f63a40a1c40 2024-03-11T20:03:29.183 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-03-11T20:03:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 msgr2=0x7f63a40a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 0x7f63a40a4100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63bc109bf0 tx=0x7f63a8005f90 comp rx=0 tx=0).stop 2024-03-11T20:03:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 msgr2=0x7f63bc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009230 tx=0x7f63b002c640 comp rx=0 tx=0).stop 2024-03-11T20:03:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 shutdown_connections 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.192+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f63a40a1c40 0x7f63a40a4100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc199800 0x7f63bc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f63bc1086a0 0x7f63bc19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 --2- 172.21.15.107:0/3247792627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc1035d0 0x7f63bc19f640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 >> 172.21.15.107:0/3247792627 conn(0x7f63bc0fee30 msgr2=0x7f63bc101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 shutdown_connections 2024-03-11T20:03:29.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.193+0000 7f63c4400640 1 -- 172.21.15.107:0/3247792627 wait complete. 2024-03-11T20:03:29.204 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-11T20:03:29.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2676188074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d0075fd0 msgr2=0x7fb9d0111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2676188074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d0075fd0 0x7fb9d0111a70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009900 tx=0x7fb9bc02bc30 comp rx=0 tx=0).stop 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2676188074 shutdown_connections 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2676188074 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2676188074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d0075fd0 0x7fb9d0111a70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2676188074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d00756b0 0x7fb9d0075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2676188074 >> 172.21.15.107:0/2676188074 conn(0x7fb9d00fede0 msgr2=0x7fb9d0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:29.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2676188074 shutdown_connections 2024-03-11T20:03:29.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.438+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2676188074 wait complete. 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.439+0000 7fb9d6c00640 1 Processor -- start 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.439+0000 7fb9d6c00640 1 -- start start 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 0x7fb9d019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d01a2c80 con 0x7fb9d00756b0 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d01a2df0 con 0x7fb9d0111fb0 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d01a2fb0 con 0x7fb9d0075fd0 2024-03-11T20:03:29.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37824/0 (socket says 172.21.15.107:37824) 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 -- 172.21.15.107:0/2954171905 learned_addr learned my addr 172.21.15.107:0/2954171905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cf400640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 0x7fb9d019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d4c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 -- 172.21.15.107:0/2954171905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 msgr2=0x7fb9d019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 0x7fb9d019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 -- 172.21.15.107:0/2954171905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 msgr2=0x7fb9d01a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cfe00640 1 -- 172.21.15.107:0/2954171905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc009560 con 0x7fb9d00756b0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9cf400640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 0x7fb9d019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.440+0000 7fb9d4c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9cfe00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0009700 con 0x7fb9d00756b0 2024-03-11T20:03:29.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9c000cd80 con 0x7fb9d00756b0 2024-03-11T20:03:29.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d01a3290 con 0x7fb9d00756b0 2024-03-11T20:03:29.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0018920 con 0x7fb9d00756b0 2024-03-11T20:03:29.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.441+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d01aab10 con 0x7fb9d00756b0 2024-03-11T20:03:29.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.442+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d00683c0 con 0x7fb9d00756b0 2024-03-11T20:03:29.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.444+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb9c0018a80 con 0x7fb9d00756b0 2024-03-11T20:03:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.445+0000 7fb9ccc00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 0x7fb9b40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.446+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7fb9c0014070 con 0x7fb9d00756b0 2024-03-11T20:03:29.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.446+0000 7fb9cf400640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 0x7fb9b40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.447+0000 7fb9cf400640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 0x7fb9b40a40e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005ab0 tx=0x7fb9bc005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:29.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.450+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9c0052d20 con 0x7fb9d00756b0 2024-03-11T20:03:29.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.648+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d010d170 con 0x7fb9d00756b0 2024-03-11T20:03:29.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.651+0000 7fb9ccc00640 1 -- 172.21.15.107:0/2954171905 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25519 (secure 0 0 0) 0x7fb9c0096a60 con 0x7fb9d00756b0 2024-03-11T20:03:29.645 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:03:29.645 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":106,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T20:03:27.712656+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/3157443745":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/2142594075":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1019473398":"2024-03-11T20:58:16.653084+0000","172.21.15.107:0/3959959887":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6801/1546798510":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/247446141":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/1546798510":"2024-03-12T19:53:25.384695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:03:29.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.653+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 msgr2=0x7fb9b40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.653+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 0x7fb9b40a40e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005ab0 tx=0x7fb9bc005a20 comp rx=0 tx=0).stop 2024-03-11T20:03:29.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.653+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 msgr2=0x7fb9d019ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.653+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9f0 tx=0x7fb9c000eeb0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 shutdown_connections 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb9b40a1c20 0x7fb9b40a40e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9d0111fb0 0x7fb9d01a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb9d0075fd0 0x7fb9d019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 --2- 172.21.15.107:0/2954171905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d00756b0 0x7fb9d019ddc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 >> 172.21.15.107:0/2954171905 conn(0x7fb9d00fede0 msgr2=0x7fb9d01135f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 shutdown_connections 2024-03-11T20:03:29.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.654+0000 7fb9d6c00640 1 -- 172.21.15.107:0/2954171905 wait complete. 2024-03-11T20:03:29.667 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T20:03:29.668 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 -- 172.21.15.107:0/2940943938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7700ffa80 msgr2=0x7fb7700fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 --2- 172.21.15.107:0/2940943938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7700ffa80 0x7fb7700fff00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb758009930 tx=0x7fb75802bc20 comp rx=0 tx=0).stop 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 -- 172.21.15.107:0/2940943938 shutdown_connections 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 --2- 172.21.15.107:0/2940943938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb770100440 0x7fb77010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 --2- 172.21.15.107:0/2940943938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7700ffa80 0x7fb7700fff00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 --2- 172.21.15.107:0/2940943938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770106160 0x7fb770106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 -- 172.21.15.107:0/2940943938 >> 172.21.15.107:0/2940943938 conn(0x7fb770075360 msgr2=0x7fb770075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:29.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 -- 172.21.15.107:0/2940943938 shutdown_connections 2024-03-11T20:03:29.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.891+0000 7fb775600640 1 -- 172.21.15.107:0/2940943938 wait complete. 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.892+0000 7fb775600640 1 Processor -- start 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.892+0000 7fb775600640 1 -- start start 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770100440 0x7fb77019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 0x7fb770197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770197eb0 con 0x7fb770106160 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770198020 con 0x7fb7700ffa80 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb775600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7701981e0 con 0x7fb770100440 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 0x7fb770197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:33328/0 (socket says 172.21.15.107:33328) 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 -- 172.21.15.107:0/2735822446 learned_addr learned my addr 172.21.15.107:0/2735822446 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb767e00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770100440 0x7fb77019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 -- 172.21.15.107:0/2735822446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770100440 msgr2=0x7fb77019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770100440 0x7fb77019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 -- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 msgr2=0x7fb770197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 0x7fb770197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76ea00640 1 -- 172.21.15.107:0/2735822446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009560 con 0x7fb7700ffa80 2024-03-11T20:03:29.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.893+0000 7fb76f400640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 0x7fb770197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb76ea00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9f0 tx=0x7fb76000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760009700 con 0x7fb7700ffa80 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76000cd80 con 0x7fb7700ffa80 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760018920 con 0x7fb7700ffa80 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7701984c0 con 0x7fb7700ffa80 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.894+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7701a4590 con 0x7fb7700ffa80 2024-03-11T20:03:29.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.895+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7700691c0 con 0x7fb7700ffa80 2024-03-11T20:03:29.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.897+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb760018a80 con 0x7fb7700ffa80 2024-03-11T20:03:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.898+0000 7fb767400640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 0x7fb7540a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.899+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7fb760014070 con 0x7fb7700ffa80 2024-03-11T20:03:29.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.899+0000 7fb767e00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 0x7fb7540a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:29.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.899+0000 7fb767e00640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 0x7fb7540a40e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb758009900 tx=0x7fb758005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:29.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:29.903+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb760052d20 con 0x7fb7700ffa80 2024-03-11T20:03:30.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:29 smithi122 ceph-mon[36397]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:30.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:29 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2954171905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:03:30.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:29 smithi145 ceph-mon[36730]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:30.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:29 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2954171905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:03:30.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:29 smithi107 ceph-mon[31712]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:30.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:29 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2954171905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:03:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.101+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7700683c0 con 0x7fb7700ffa80 2024-03-11T20:03:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.102+0000 7fb767400640 1 -- 172.21.15.107:0/2735822446 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb760096a60 con 0x7fb7700ffa80 2024-03-11T20:03:30.096 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 msgr2=0x7fb7540a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 0x7fb7540a40e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb758009900 tx=0x7fb758005a20 comp rx=0 tx=0).stop 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 msgr2=0x7fb77019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9f0 tx=0x7fb76000eeb0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 shutdown_connections 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb7540a1c20 0x7fb7540a40e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb770106160 0x7fb770197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb770100440 0x7fb77019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 --2- 172.21.15.107:0/2735822446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7700ffa80 0x7fb77019d610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 >> 172.21.15.107:0/2735822446 conn(0x7fb770075360 msgr2=0x7fb7700fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.105+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 shutdown_connections 2024-03-11T20:03:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.106+0000 7fb775600640 1 -- 172.21.15.107:0/2735822446 wait complete. 2024-03-11T20:03:30.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 -- 172.21.15.107:0/2786919351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e4107720 msgr2=0x7f12e4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 --2- 172.21.15.107:0/2786919351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e4107720 0x7f12e4107ba0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009930 tx=0x7f12cc02bc30 comp rx=0 tx=0).stop 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 -- 172.21.15.107:0/2786919351 shutdown_connections 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 --2- 172.21.15.107:0/2786919351 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e41080e0 0x7f12e41145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 --2- 172.21.15.107:0/2786919351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e4107720 0x7f12e4107ba0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 --2- 172.21.15.107:0/2786919351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e410d730 0x7f12e410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 -- 172.21.15.107:0/2786919351 >> 172.21.15.107:0/2786919351 conn(0x7f12e40756c0 msgr2=0x7f12e4077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 -- 172.21.15.107:0/2786919351 shutdown_connections 2024-03-11T20:03:30.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.334+0000 7f12e8a00640 1 -- 172.21.15.107:0/2786919351 wait complete. 2024-03-11T20:03:30.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.335+0000 7f12e8a00640 1 Processor -- start 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.335+0000 7f12e8a00640 1 -- start start 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e4107720 0x7f12e419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a2cf0 con 0x7f12e410d730 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a2e60 con 0x7f12e41080e0 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41a3020 con 0x7f12e4107720 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:33352/0 (socket says 172.21.15.107:33352) 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37864/0 (socket says 172.21.15.107:37864) 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 -- 172.21.15.107:0/1310600612 learned_addr learned my addr 172.21.15.107:0/1310600612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e1e00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e4107720 0x7f12e419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 -- 172.21.15.107:0/1310600612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e4107720 msgr2=0x7f12e419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e4107720 0x7f12e419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 -- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 msgr2=0x7f12e41a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12d9400640 1 -- 172.21.15.107:0/1310600612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009560 con 0x7f12e41080e0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.336+0000 7f12e2800640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12d9400640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc02cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc004350 con 0x7f12e41080e0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e41a32a0 con 0x7f12e41080e0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12cc0044b0 con 0x7f12e41080e0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc003d30 con 0x7f12e41080e0 2024-03-11T20:03:30.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.337+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e41aab80 con 0x7f12e41080e0 2024-03-11T20:03:30.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.338+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e40683c0 con 0x7f12e41080e0 2024-03-11T20:03:30.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.340+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f12cc004df0 con 0x7f12e41080e0 2024-03-11T20:03:30.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.341+0000 7f12db400640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 0x7f12c80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.342+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f12cc0f0620 con 0x7f12e41080e0 2024-03-11T20:03:30.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.342+0000 7f12e1e00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 0x7f12c80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.342+0000 7f12e1e00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 0x7f12c80a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009b50 tx=0x7f12d4005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:30.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.344+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12cc0b9280 con 0x7f12e41080e0 2024-03-11T20:03:30.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.539+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12e410fc90 con 0x7f12e41080e0 2024-03-11T20:03:30.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.539+0000 7f12db400640 1 -- 172.21.15.107:0/1310600612 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12cc0bc940 con 0x7f12e41080e0 2024-03-11T20:03:30.533 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.542+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 msgr2=0x7f12c80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.542+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 0x7f12c80a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009b50 tx=0x7f12d4005f90 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.542+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 msgr2=0x7f12e419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.542+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc02cd80 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 shutdown_connections 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f12c80a1ba0 0x7f12c80a4060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12e41080e0 0x7f12e419e370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e410d730 0x7f12e41a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 --2- 172.21.15.107:0/1310600612 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e4107720 0x7f12e419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 >> 172.21.15.107:0/1310600612 conn(0x7f12e40756c0 msgr2=0x7f12e4075ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 shutdown_connections 2024-03-11T20:03:30.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.543+0000 7f12e8a00640 1 -- 172.21.15.107:0/1310600612 wait complete. 2024-03-11T20:03:30.554 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 -- 172.21.15.107:0/1685503519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101d30 msgr2=0x7f89b4105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 --2- 172.21.15.107:0/1685503519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101d30 0x7f89b4105d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f899c009930 tx=0x7f899c02bc30 comp rx=0 tx=0).stop 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 -- 172.21.15.107:0/1685503519 shutdown_connections 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 --2- 172.21.15.107:0/1685503519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 --2- 172.21.15.107:0/1685503519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101d30 0x7f89b4105d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 --2- 172.21.15.107:0/1685503519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4101380 0x7f89b4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 -- 172.21.15.107:0/1685503519 >> 172.21.15.107:0/1685503519 conn(0x7f89b4078580 msgr2=0x7f89b4078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:30.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 -- 172.21.15.107:0/1685503519 shutdown_connections 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.781+0000 7f89bb400640 1 -- 172.21.15.107:0/1685503519 wait complete. 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 Processor -- start 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 -- start start 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 0x7f89b419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4199820 0x7f89b4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b419a2d0 con 0x7f89b4101380 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b419a440 con 0x7f89b4199820 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.782+0000 7f89bb400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b419a5b0 con 0x7f89b4106450 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89b8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 0x7f89b419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:51366/0 (socket says 172.21.15.107:51366) 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 -- 172.21.15.107:0/2578650000 learned_addr learned my addr 172.21.15.107:0/2578650000 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:30.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89b9400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4199820 0x7f89b4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 -- 172.21.15.107:0/2578650000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4199820 msgr2=0x7f89b4199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4199820 0x7f89b4199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 -- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 msgr2=0x7f89b419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 0x7f89b419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 -- 172.21.15.107:0/2578650000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c009560 con 0x7f89b4106450 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89b8a00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 0x7f89b419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.783+0000 7f89abe00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f899c005ab0 tx=0x7f899c02c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.784+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0042d0 con 0x7f89b4106450 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.784+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b41a6500 con 0x7f89b4106450 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.784+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f899c004430 con 0x7f89b4106450 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.784+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c003be0 con 0x7f89b4106450 2024-03-11T20:03:30.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.784+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b41a69e0 con 0x7f89b4106450 2024-03-11T20:03:30.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.785+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b4102850 con 0x7f89b4106450 2024-03-11T20:03:30.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.787+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f899c003d40 con 0x7f89b4106450 2024-03-11T20:03:30.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.788+0000 7f89b2000640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 0x7f89980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:30.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.788+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f899c0f0190 con 0x7f89b4106450 2024-03-11T20:03:30.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.789+0000 7f89b8a00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 0x7f89980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:30.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.789+0000 7f89b8a00640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 0x7f89980a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005fd0 tx=0x7f89a4009500 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:30.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.792+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f899c0b8d00 con 0x7f89b4106450 2024-03-11T20:03:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.991+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89b4101d30 con 0x7f89b4106450 2024-03-11T20:03:30.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.992+0000 7f89b2000640 1 -- 172.21.15.107:0/2578650000 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f899c0bc3c0 con 0x7f89b4106450 2024-03-11T20:03:30.986 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T20:03:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.994+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 msgr2=0x7f89980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.994+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 0x7f89980a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005fd0 tx=0x7f89a4009500 comp rx=0 tx=0).stop 2024-03-11T20:03:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.994+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 msgr2=0x7f89b419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.994+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f899c005ab0 tx=0x7f899c02c5f0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 shutdown_connections 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f89980a1ba0 0x7f89980a4060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b4199820 0x7f89b4199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89b4106450 0x7f89b419fbf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 --2- 172.21.15.107:0/2578650000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89b4101380 0x7f89b419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 >> 172.21.15.107:0/2578650000 conn(0x7f89b4078580 msgr2=0x7f89b40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 shutdown_connections 2024-03-11T20:03:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:30.995+0000 7f89bb400640 1 -- 172.21.15.107:0/2578650000 wait complete. 2024-03-11T20:03:31.001 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:30 smithi122 ceph-mon[36397]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:31.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:30 smithi122 ceph-mon[36397]: pgmap v315: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:31.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:30 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2735822446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:30 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1310600612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:30 smithi107 ceph-mon[31712]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:30 smithi107 ceph-mon[31712]: pgmap v315: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:30 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2735822446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:30 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1310600612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T20:03:31.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:30 smithi145 ceph-mon[36730]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T20:03:31.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:30 smithi145 ceph-mon[36730]: pgmap v315: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:31.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:30 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2735822446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:30 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1310600612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:03:31.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 -- 172.21.15.107:0/348260528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 msgr2=0x7f2634107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 --2- 172.21.15.107:0/348260528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f2634107ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f261c009860 tx=0x7f261c02bb70 comp rx=0 tx=0).stop 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 -- 172.21.15.107:0/348260528 shutdown_connections 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 --2- 172.21.15.107:0/348260528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f2634107ba0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 --2- 172.21.15.107:0/348260528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26341080e0 0x7f26341145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 --2- 172.21.15.107:0/348260528 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f263410d730 0x7f263410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 -- 172.21.15.107:0/348260528 >> 172.21.15.107:0/348260528 conn(0x7f26340756c0 msgr2=0x7f2634077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 -- 172.21.15.107:0/348260528 shutdown_connections 2024-03-11T20:03:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.226+0000 7f2639200640 1 -- 172.21.15.107:0/348260528 wait complete. 2024-03-11T20:03:31.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 Processor -- start 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 -- start start 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26341080e0 0x7f263419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 0x7f26341a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341a2c60 con 0x7f263410d730 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341a2dd0 con 0x7f2634107720 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.227+0000 7f2639200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341a2f90 con 0x7f26341080e0 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2632800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 0x7f26341a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2632800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 0x7f26341a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37904/0 (socket says 172.21.15.107:37904) 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2629400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26341080e0 0x7f263419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:33380/0 (socket says 172.21.15.107:33380) 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2632800640 1 -- 172.21.15.107:0/1431951750 learned_addr learned my addr 172.21.15.107:0/1431951750 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 -- 172.21.15.107:0/1431951750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26341080e0 msgr2=0x7f263419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26341080e0 0x7f263419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 -- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 msgr2=0x7f26341a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 0x7f26341a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 -- 172.21.15.107:0/1431951750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009490 con 0x7f2634107720 2024-03-11T20:03:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.228+0000 7f2631e00640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2624009fd0 tx=0x7f262400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.229+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400cea0 con 0x7f2634107720 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.229+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2624005020 con 0x7f2634107720 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.229+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26341a3270 con 0x7f2634107720 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.229+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26240106d0 con 0x7f2634107720 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.229+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341aaaf0 con 0x7f2634107720 2024-03-11T20:03:31.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.230+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26340683c0 con 0x7f2634107720 2024-03-11T20:03:31.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.232+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f2624005190 con 0x7f2634107720 2024-03-11T20:03:31.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.233+0000 7f262b400640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 0x7f26180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.233+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f2624014070 con 0x7f2634107720 2024-03-11T20:03:31.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.233+0000 7f2629400640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 0x7f26180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.234+0000 7f2629400640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 0x7f26180a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f261c02c180 tx=0x7f261c0056d0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:31.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.235+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2624096d30 con 0x7f2634107720 2024-03-11T20:03:31.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.438+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f263410fc90 con 0x7f2634107720 2024-03-11T20:03:31.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.439+0000 7f262b400640 1 -- 172.21.15.107:0/1431951750 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f262409a3f0 con 0x7f2634107720 2024-03-11T20:03:31.433 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 msgr2=0x7f26180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 0x7f26180a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f261c02c180 tx=0x7f261c0056d0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 msgr2=0x7f263419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2624009fd0 tx=0x7f262400ef90 comp rx=0 tx=0).stop 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 shutdown_connections 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f26180a1ba0 0x7f26180a4060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263410d730 0x7f26341a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f26341080e0 0x7f263419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 --2- 172.21.15.107:0/1431951750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634107720 0x7f263419dda0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.441+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 >> 172.21.15.107:0/1431951750 conn(0x7f26340756c0 msgr2=0x7f2634075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:31.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.442+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 shutdown_connections 2024-03-11T20:03:31.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.442+0000 7f2639200640 1 -- 172.21.15.107:0/1431951750 wait complete. 2024-03-11T20:03:31.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-11T20:03:31.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 -- 172.21.15.107:0/647148582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d0111fa0 msgr2=0x7fb0d0114390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/647148582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d0111fa0 0x7fb0d0114390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009c10 tx=0x7fb0c802bee0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 -- 172.21.15.107:0/647148582 shutdown_connections 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/647148582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d0111fa0 0x7fb0d0114390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/647148582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0075fd0 0x7fb0d0111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/647148582 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d00756b0 0x7fb0d0075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 -- 172.21.15.107:0/647148582 >> 172.21.15.107:0/647148582 conn(0x7fb0d00fedd0 msgr2=0x7fb0d01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 -- 172.21.15.107:0/647148582 shutdown_connections 2024-03-11T20:03:31.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.670+0000 7fb0d4e00640 1 -- 172.21.15.107:0/647148582 wait complete. 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 Processor -- start 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 -- start start 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 0x7fb0d019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0075fd0 0x7fb0d019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01a2c20 con 0x7fb0d0111fa0 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01a2d90 con 0x7fb0d00756b0 2024-03-11T20:03:31.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.671+0000 7fb0d4e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01a2f50 con 0x7fb0d0075fd0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37910/0 (socket says 172.21.15.107:37910) 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 -- 172.21.15.107:0/2120209958 learned_addr learned my addr 172.21.15.107:0/2120209958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0c5400640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0075fd0 0x7fb0d019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0cde00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 0x7fb0d019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 -- 172.21.15.107:0/2120209958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0075fd0 msgr2=0x7fb0d019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0075fd0 0x7fb0d019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 -- 172.21.15.107:0/2120209958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 msgr2=0x7fb0d019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 0x7fb0d019dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 -- 172.21.15.107:0/2120209958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009560 con 0x7fb0d0111fa0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0cde00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 0x7fb0d019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.672+0000 7fb0ce800640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8036040 tx=0x7fb0c8033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:31.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.673+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8003b80 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.673+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c8009840 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.673+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c8003ce0 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.673+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c803e890 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.673+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01aa8d0 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.674+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0068430 con 0x7fb0d0111fa0 2024-03-11T20:03:31.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.676+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fb0c803e9f0 con 0x7fb0d0111fa0 2024-03-11T20:03:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.677+0000 7fb0c7400640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 0x7fb0b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:03:31.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.677+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9561+0+0 (secure 0 0 0) 0x7fb0c80f0880 con 0x7fb0d0111fa0 2024-03-11T20:03:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.677+0000 7fb0cde00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 0x7fb0b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:03:31.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.678+0000 7fb0cde00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 0x7fb0b40a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fc0 tx=0x7fb0c0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:03:31.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.681+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0c80b9450 con 0x7fb0d0111fa0 2024-03-11T20:03:31.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.936+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fb0d010d130 con 0x7fb0d0111fa0 2024-03-11T20:03:31.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.967+0000 7fb0c7400640 1 -- 172.21.15.107:0/2120209958 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+735010 (secure 0 0 0) 0x7fb0c8033870 con 0x7fb0d0111fa0 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 26, 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 15324, 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2024-03-11T20:03:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4104906604 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4104906604 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/4104906604", 2024-03-11T20:03:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2024-03-11T19:53:25.384937+0000", 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540701547738038271, 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25436, 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-11T20:03:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-11T20:03:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-11T20:03:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-11T20:03:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-11T20:03:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-11T20:03:31.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-11T20:03:31.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-11T20:03:31.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-11T20:03:31.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-11T20:03:31.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:31.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:31.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.000 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.003 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-11T20:03:32.004 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.006 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.011 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.015 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.016 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-11T20:03:32.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.018 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-11T20:03:32.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-11T20:03:32.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T20:03:32.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T20:03:32.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T20:03:32.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-11T20:03:32.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.041 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-11T20:03:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-11T20:03:32.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-11T20:03:32.051 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-11T20:03:32.061 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T20:03:32.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T20:03:32.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-11T20:03:32.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.073 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.077 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.078 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.081 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T20:03:32.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.105 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T20:03:32.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T20:03:32.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.120 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T20:03:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T20:03:32.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T20:03:32.124 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.130 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-11T20:03:32.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-11T20:03:32.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.134 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.136 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.140 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.141 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.143 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.146 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.147 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.148 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-11T20:03:32.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.152 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.154 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.156 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-11T20:03:32.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T20:03:32.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.165 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.166 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.169 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-11T20:03:32.171 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-11T20:03:32.175 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-11T20:03:32.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-11T20:03:32.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-11T20:03:32.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.184 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-11T20:03:32.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.192 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.193 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.194 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-11T20:03:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-11T20:03:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T20:03:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-11T20:03:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T20:03:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-11T20:03:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-11T20:03:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T20:03:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.306 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.307 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.310 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.319 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.321 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.322 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.337 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.338 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-11T20:03:32.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-11T20:03:32.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.347 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.348 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T20:03:32.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.351 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.359 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.362 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.364 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.366 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.374 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.378 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-11T20:03:32.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-11T20:03:32.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.390 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.392 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.397 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.398 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.399 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.404 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.410 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-11T20:03:32.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.412 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.413 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.420 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25447, 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.426 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.427 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.430 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.442 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-11T20:03:32.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:32.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T20:03:32.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T20:03:32.450 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:32.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.452 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.455 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-11T20:03:32.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T20:03:32.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T20:03:32.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-11T20:03:32.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T20:03:32.472 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.473 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.474 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-11T20:03:32.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.481 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-11T20:03:32.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.486 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.489 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-11T20:03:32.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.492 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.505 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T20:03:32.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.508 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T20:03:32.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.512 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T20:03:32.515 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T20:03:32.516 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T20:03:32.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T20:03:32.520 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T20:03:32.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.522 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T20:03:32.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T20:03:32.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T20:03:32.525 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.527 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.528 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.532 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-11T20:03:32.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T20:03:32.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.539 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.540 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T20:03:32.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T20:03:32.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.548 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.550 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.551 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.557 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.558 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.559 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.560 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.562 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.563 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.565 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.567 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.571 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.572 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T20:03:32.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T20:03:32.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.583 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.585 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.589 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.590 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.592 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.593 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.598 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:32.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.602 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.603 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.605 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.609 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T20:03:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T20:03:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-11T20:03:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.616 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.617 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.621 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.622 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.626 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.627 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.628 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-11T20:03:32.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.630 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.632 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.633 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.635 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:32.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:32.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-11T20:03:32.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.654 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-11T20:03:32.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-11T20:03:32.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.661 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-11T20:03:32.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.664 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.665 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-11T20:03:32.671 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-11T20:03:32.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.676 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.677 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.681 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.682 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.683 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.684 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-11T20:03:32.688 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.689 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.695 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.696 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.699 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.702 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.703 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.705 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.707 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.709 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.712 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.718 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-11T20:03:32.722 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.725 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.726 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.727 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.729 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.731 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-11T20:03:32.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.734 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.737 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.738 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-11T20:03:32.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:32.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-11T20:03:32.747 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.749 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.750 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.753 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:32.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.763 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-11T20:03:32.767 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.769 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.770 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.771 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.775 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.778 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-11T20:03:32.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.783 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.786 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-11T20:03:32.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.801 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.805 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.808 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.816 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.817 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.818 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.825 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-11T20:03:32.827 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.830 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.833 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.835 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.837 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.840 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.843 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.846 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-11T20:03:32.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.851 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.856 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.859 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.862 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-11T20:03:32.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T20:03:32.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.879 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "restful", 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_schedule" 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.880 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.885 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-11T20:03:32.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.896 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T20:03:32.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-11T20:03:32.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T20:03:32.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T20:03:32.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.905 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-11T20:03:32.911 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.916 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.918 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-11T20:03:32.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.922 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.924 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-11T20:03:32.926 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-11T20:03:32.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.951 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-11T20:03:32.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T20:03:32.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T20:03:32.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T20:03:32.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-11T20:03:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-11T20:03:32.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.982 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-11T20:03:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.986 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:32.989 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:32.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-11T20:03:32.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.994 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:32.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T20:03:33.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.005 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.006 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.016 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T20:03:33.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T20:03:33.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T20:03:33.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.026 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T20:03:33.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-11T20:03:33.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.033 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.036 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-11T20:03:33.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.045 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:33.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.056 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.061 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-11T20:03:33.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-11T20:03:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-11T20:03:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-11T20:03:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-11T20:03:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.076 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.078 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-11T20:03:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.084 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.087 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.089 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.090 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-11T20:03:33.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:33.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.102 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.105 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.108 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T20:03:33.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.115 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.117 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T20:03:33.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.119 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.120 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.121 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.122 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.134 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.138 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.139 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.140 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.142 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.144 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:33.146 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.148 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.149 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.151 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-11T20:03:33.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.155 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.156 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.159 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.161 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.165 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.167 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.168 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.174 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.182 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-11T20:03:33.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-11T20:03:33.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T20:03:33.189 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.191 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.194 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.204 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.206 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-11T20:03:33.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.215 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-11T20:03:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.222 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.224 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.233 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.239 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.244 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.245 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-11T20:03:33.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.248 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.253 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T20:03:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-11T20:03:33.255 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.256 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.259 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.263 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.268 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.276 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.278 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T20:03:33.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.283 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.286 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-11T20:03:33.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.292 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.298 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T20:03:33.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.303 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.312 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-11T20:03:33.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-11T20:03:33.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-11T20:03:33.325 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-11T20:03:33.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-11T20:03:33.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-11T20:03:33.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2024-03-11T20:03:33.331 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-11T20:03:33.332 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-11T20:03:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-11T20:03:33.334 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "squid": [ 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-11T20:03:33.335 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 96, 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 898821402 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2024-03-11T20:03:33.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1678999227 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.337 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3272730844 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 788892690 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-11T20:03:33.338 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1461392000 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3356835229 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.339 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:33.340 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-11T20:03:33.340 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T20:03:33.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.976+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 msgr2=0x7fb0b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:33.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.976+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 0x7fb0b40a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009fc0 tx=0x7fb0c0005f90 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.976+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 msgr2=0x7fb0d01a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.976+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8036040 tx=0x7fb0c8033680 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 shutdown_connections 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fb0b40a1ba0 0x7fb0b40a4060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0d0111fa0 0x7fb0d01a2620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb0d0075fd0 0x7fb0d019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 --2- 172.21.15.107:0/2120209958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d00756b0 0x7fb0d019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 >> 172.21.15.107:0/2120209958 conn(0x7fb0d00fedd0 msgr2=0x7fb0d01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 shutdown_connections 2024-03-11T20:03:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:03:31.977+0000 7fb0d4e00640 1 -- 172.21.15.107:0/2120209958 wait complete. 2024-03-11T20:03:33.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:31 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2578650000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:31 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1431951750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:31 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2578650000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:31 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1431951750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:32 smithi145 ceph-mon[36730]: pgmap v316: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:32 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2120209958' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:31 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2578650000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.343 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:31 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1431951750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:03:33.344 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:32 smithi122 ceph-mon[36397]: pgmap v316: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-11T20:03:33.344 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:32 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2120209958' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T20:03:33.349 DEBUG:tasks.check_counter:Getting stats from a 2024-03-11T20:03:33.350 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-11T20:03:33.350 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:32 smithi107 ceph-mon[31712]: pgmap v316: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-03-11T20:03:33.350 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:32 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2120209958' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T20:03:33.802 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T20:03:34.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:34 smithi107 ceph-mon[31712]: pgmap v317: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:35.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:34 smithi122 ceph-mon[36397]: pgmap v317: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:35.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:34 smithi145 ceph-mon[36730]: pgmap v317: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:36.333 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T20:03:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 50350, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 84283, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 18972602, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 85802991, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 16, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 14.854568463, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 9.738176023, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 6.641522216, 2024-03-11T20:03:36.334 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.394866699, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 84271, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 11.430921936, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000135644 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 93447, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.047695466, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000510 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 17141280, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 43353184 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.335 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 13557, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 13987, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 5837815, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 38607341, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 7, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 2.355769994, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 1.042824248, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 1.163993221, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.308909642, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 13987, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.464430718, 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000176194 2024-03-11T20:03:36.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 14776, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007066077, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000478 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1656528, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 154512 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 17073, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 19909, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 9693147, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 63918862, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 11, 2024-03-11T20:03:36.337 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 3.301823096, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 1.503676695, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 1.289290315, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.568260560, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 19906, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.195785973, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000160543 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 17039, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007350697, 2024-03-11T20:03:36.338 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000431 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2464, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 453776 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-11T20:03:36.339 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4567, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 70.697900259, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.015480162 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 15052, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 7.208343333, 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000478896 2024-03-11T20:03:36.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 23999, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 23863, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 23863, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 405.548508659, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.016994866 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-03-11T20:03:36.341 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 136, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 15, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 71, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 7, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 22951, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 10, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 13041, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 9900, 2024-03-11T20:03:36.342 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 22931, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 1786, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 2, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 37, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 30396, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 29256, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 86, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 343, 2024-03-11T20:03:36.343 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 3, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 6, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 101, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 1362, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 306, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 4744, 2024-03-11T20:03:36.344 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 6, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 12118, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 1788379046, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 3, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:03:36.345 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 119, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 3942, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 19141, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 11, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 18814, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 7754, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 155, 2024-03-11T20:03:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 13298 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 349, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 150, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 991, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 10133, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 9136, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 11, 2024-03-11T20:03:36.347 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 5, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 101, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 7, 2024-03-11T20:03:36.348 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 1, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 52269, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 52269, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 70, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 70, 2024-03-11T20:03:36.349 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 6, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 141726068, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3140, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 389.702676212, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.124109132 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-03-11T20:03:36.350 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 11105, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 21548, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 10443, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1077, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1396, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 319, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 22951, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 33926, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 10975, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 2, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 13157, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 13155, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 164308, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 207228 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.351 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 29721, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 525, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 23999, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 194, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 1115, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6737, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 279.642161233, 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.041508410 2024-03-11T20:03:36.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 66, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.212767505, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.003223750 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 18, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.252423750, 2024-03-11T20:03:36.353 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.014023541 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4861, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.219006844, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001073648 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-03-11T20:03:36.354 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000098719, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000098719 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.355 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.356 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.356 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-03-11T20:03:36.356 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.356 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 782, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 13.831629025, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.017687505 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 47, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.093831052, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001996405 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.357 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 41, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.026634156, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000649613 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1041, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.375367783, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001321198 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 49, 2024-03-11T20:03:36.358 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 28.571686669, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.583095646 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1027, 2024-03-11T20:03:36.359 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 52.749051623, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.051362270 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 12, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.327731837, 2024-03-11T20:03:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.027310986 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.361 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 41, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.088862335, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002167374 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-03-11T20:03:36.362 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 9140, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 23.157256128, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002533616 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-03-11T20:03:36.363 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 5, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 3, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 9946, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 4973, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 587, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:36.364 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:36.365 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:36.366 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 8617445, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 12297, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 1144, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 13, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9816, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 205, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 44722898, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 775470, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 22422, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 141412321, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 27, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 22422, 2024-03-11T20:03:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 22422, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 162.029342885, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.007226355 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 22422, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 22843 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 22422, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 304, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 22118, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-03-11T20:03:36.369 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 285, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 3425, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 140, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 6, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 7, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 18263, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 15, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-03-11T20:03:36.370 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 11, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-03-11T20:03:36.371 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 106, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 16, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-03-11T20:03:36.372 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 485, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 34, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 142 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 3 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-03-11T20:03:36.373 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 21, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 12, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 18225, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 9136, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 80976, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 25182602, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 80976, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 80976, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 25182602, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-11T20:03:36.375 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 22422, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 142586872, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 22422, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 142586872, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:36.376 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 22422, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 22422, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 22422, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 22422, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-11T20:03:36.377 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 9136, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1174960, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 9136, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 285, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1174960, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x561b0a4c06e0": { 2024-03-11T20:03:36.378 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 52269, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 137531764, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 52269, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3140, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 137531764, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:36.379 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T20:03:37.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:36 smithi122 ceph-mon[36397]: pgmap v318: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:37.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:36 smithi145 ceph-mon[36730]: pgmap v318: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:37.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:36 smithi107 ceph-mon[31712]: pgmap v318: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:37.331 INFO:tasks.check_counter:Daemon mds.a mds.exported=101 2024-03-11T20:03:37.331 INFO:tasks.check_counter:Daemon mds.a mds.imported=306 2024-03-11T20:03:37.331 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=3 2024-03-11T20:03:37.332 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-11T20:03:37.332 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T20:03:37.332 DEBUG:tasks.check_counter:Getting stats from d 2024-03-11T20:03:37.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-11T20:03:37.769 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T20:03:39.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:38 smithi122 ceph-mon[36397]: pgmap v319: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:39.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:38 smithi145 ceph-mon[36730]: pgmap v319: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:39.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:38 smithi107 ceph-mon[31712]: pgmap v319: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3024, 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-03-11T20:03:40.094 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.004309414, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000070697, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001339152, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001463912, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000487970 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.095 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 240, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 229, 2024-03-11T20:03:40.096 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1568544, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 137888, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.120847396, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.031995770, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.114878214, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.013637287, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 228, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.021717606, 2024-03-11T20:03:40.097 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000095252 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1568544, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 137888 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-03-11T20:03:40.098 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 175, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 185600, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.035201006, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023963512, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.005121705, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 172, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.035580727, 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000206864 2024-03-11T20:03:40.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 185600 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-11T20:03:40.100 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:40.101 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:40.102 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148731, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 31, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:40.103 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-11T20:03:40.104 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 242, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1534704, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:40.105 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 242, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 242, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1534704, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:40.106 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T20:03:40.917 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-11T20:03:40.917 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-11T20:03:40.917 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-11T20:03:40.918 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-11T20:03:40.918 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-11T20:03:40.918 DEBUG:tasks.check_counter:Getting stats from g 2024-03-11T20:03:40.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-11T20:03:41.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:40 smithi107 ceph-mon[31712]: pgmap v320: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:41.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:40 smithi145 ceph-mon[36730]: pgmap v320: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:41.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:40 smithi122 ceph-mon[36397]: pgmap v320: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:41.369 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T20:03:43.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:42 smithi145 ceph-mon[36730]: pgmap v321: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-11T20:03:43.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:42 smithi107 ceph-mon[31712]: pgmap v321: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-11T20:03:43.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:42 smithi122 ceph-mon[36397]: pgmap v321: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 236, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 225, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1567456, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 135392, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.100491705, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.034169927, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.077200583, 2024-03-11T20:03:43.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.009623692, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 224, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.023150856, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000103352 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1567456, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 135392 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6, 2024-03-11T20:03:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 174, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3344, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 185520, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.033858456, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023762655, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004207188, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 172, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.033891833, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000197045 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 185520 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000707522, 2024-03-11T20:03:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000526266, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:03:43.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:43.724 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:43.725 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148714, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-11T20:03:43.726 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:43.727 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 238, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1534184, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 238, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 238, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1534184, 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:43.728 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:43.729 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:43.729 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:43.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:43.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:43.729 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T20:03:44.638 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-11T20:03:44.638 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-11T20:03:44.638 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-11T20:03:44.638 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-11T20:03:44.638 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-11T20:03:44.639 DEBUG:tasks.check_counter:Getting stats from j 2024-03-11T20:03:44.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-11T20:03:45.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:44 smithi145 ceph-mon[36730]: pgmap v322: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:45.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:44 smithi107 ceph-mon[31712]: pgmap v322: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:45.087 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T20:03:45.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:44 smithi122 ceph-mon[36397]: pgmap v322: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-11T20:03:46.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:46 smithi107 ceph-mon[31712]: pgmap v323: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:47.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:46 smithi145 ceph-mon[36730]: pgmap v323: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:47.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:46 smithi122 ceph-mon[36397]: pgmap v323: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:47.432 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-11T20:03:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003904678, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001043111, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 246, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 233, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1572112, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 139584, 2024-03-11T20:03:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.120606129, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.023753008, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.136902637, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.010400704, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 233, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.023726382, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000101829 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1572112, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 139584 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 178, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 186128, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.035189875, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.025785487, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003346965, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 175, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.030025593, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000171574 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 186128 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-11T20:03:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149745, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 31, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-11T20:03:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 244, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1534964, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 244, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 244, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1534964, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-11T20:03:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-11T20:03:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-11T20:03:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-11T20:03:47.443 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-11T20:03:48.383 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-11T20:03:48.383 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-11T20:03:48.383 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-11T20:03:48.383 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-11T20:03:48.383 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-11T20:03:48.383 DEBUG:tasks.check_counter:Getting stats from b 2024-03-11T20:03:48.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-11T20:03:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T20:03:49.027 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:48 smithi122 ceph-mon[36397]: pgmap v324: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:49.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:48 smithi107 ceph-mon[31712]: pgmap v324: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:49.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:48 smithi145 ceph-mon[36730]: pgmap v324: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:50.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:49 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:03:50.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:49 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:49 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.052 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:49 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:03:50.052 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:49 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.052 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:49 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:49 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T20:03:50.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:49 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:49 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 13965, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 12980, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 4796490, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 29944301, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 15, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 1.063270230, 2024-03-11T20:03:50.443 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.407876199, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.490203931, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.233939069, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 12970, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 4.397366820, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000339041 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 14925, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.002334696, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000156 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 313072, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 87024 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.444 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 17892, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 26443, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 9336762, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 27544656, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 56, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 7, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 1.430207693, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.614987523, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.813684488, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.173197515, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 26442, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 5.659440262, 2024-03-11T20:03:50.445 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000214032 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 25841, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.004406736, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000170 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 7710624, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 14443072 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 11029, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 11797, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 4115833, 2024-03-11T20:03:50.446 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 20372149, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 15, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 7, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.766332407, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.315739552, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.332340728, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.135246886, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 11792, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 3.819494700, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000323905 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 13810, 2024-03-11T20:03:50.447 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.002101196, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000152 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 43472, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 472816 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 6862, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 9.238924675, 2024-03-11T20:03:50.448 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001346389 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 3749, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.246219338, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000065676 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 7162, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 7038, 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-03-11T20:03:50.449 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 7038, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 50.873978031, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.007228470 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 124, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 6, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 1694, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 3, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 4298, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 2799, 2024-03-11T20:03:50.450 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 1218, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 281, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 299, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 1548, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 3, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 3, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 14, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 10701, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 9761, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 121, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 179, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 2, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 2, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 15, 2024-03-11T20:03:50.451 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 91, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 846, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 73, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 742, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 6, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:03:50.452 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 70, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 2431, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 4898, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 219, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 5141, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 1805, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 111, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 4291 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-03-11T20:03:50.453 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 189, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 195, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 424, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 2081, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 1622, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 93, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 5, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-03-11T20:03:50.454 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 94, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 3, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 12394, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 12340, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 12340, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 54, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 4, 2024-03-11T20:03:50.455 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 38, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 37, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 37, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 1, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 1, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 1, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 46139146, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 46300470, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 0, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 3742, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 66.542308538, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.017782551 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 0 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.456 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1777, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 4300, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 2523, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 341, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 646, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 305, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 4298, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 8044, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 3746, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 3, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 2653, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 2650, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 118020, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215420 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-03-11T20:03:50.457 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 10096, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 972, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 7162, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 280, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 1397, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 782, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 1.936977327, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002476953 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 213, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.449344879, 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002109600 2024-03-11T20:03:50.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 200, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.819249053, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004096245 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1196, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 10.935008774, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.009142983 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-03-11T20:03:50.459 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000194773, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000194773 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.460 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-03-11T20:03:50.461 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 10, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.009305560, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000930556 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 388, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.885320120, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002281752 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 354, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.710572564, 2024-03-11T20:03:50.462 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002007267 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 10, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 1.954752148, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.195475214 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 6, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.013137188, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002189531 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 455, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.981778816, 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002157755 2024-03-11T20:03:50.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 465, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 15.925465815, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.034248313 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.464 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 510, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 10.824941825, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.021225376 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 4, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.028515259, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.007128814 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 227, 2024-03-11T20:03:50.465 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.898293720, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.003957241 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-03-11T20:03:50.466 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 9, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.024225134, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002691681 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 377, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.883757226, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002344183 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1831, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 3.593137850, 2024-03-11T20:03:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001962390 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 2, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 7, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 5, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 2, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-03-11T20:03:50.468 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 117, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 58, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 612, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 960, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 960, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:50.469 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:50.470 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 3343932, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 2548, 2024-03-11T20:03:50.471 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 880, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 10, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9816, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 205, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 8880453, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 812320, 2024-03-11T20:03:50.472 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 13819, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 52770156, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 31, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 13819, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 13819, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 532.576498184, 2024-03-11T20:03:50.473 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.038539438 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 13819, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 24485 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 13819, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 175, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 13644, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 1248, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 3889, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 141, 2024-03-11T20:03:50.474 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 3883, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 188, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 56, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 3, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 3499, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 6, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 7149, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-03-11T20:03:50.475 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 265, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-03-11T20:03:50.476 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 106, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 22, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 12, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 12, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 3408, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 40, 2024-03-11T20:03:50.477 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 710 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 1, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 3 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 23, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-03-11T20:03:50.478 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 13, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 3023, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 1622, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 42882, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 13716283, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 42882, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 42882, 2024-03-11T20:03:50.479 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 13716283, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:50.480 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 13819, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 52957433, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 13819, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 52957433, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-03-11T20:03:50.481 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 13819, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 13819, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 13819, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 13819, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.482 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 1622, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 188270, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 1622, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 141, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 188270, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.483 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x55d7d709a6e0": { 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 12394, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 42106166, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 12394, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 3742, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 42106166, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:50.484 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:50.485 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:50.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:50.485 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-11T20:03:51.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:50 smithi122 ceph-mon[36397]: pgmap v325: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:51.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:50 smithi145 ceph-mon[36730]: pgmap v325: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:51.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:50 smithi107 ceph-mon[31712]: pgmap v325: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:51.299 INFO:tasks.check_counter:Daemon mds.b mds.exported=91 2024-03-11T20:03:51.299 INFO:tasks.check_counter:Daemon mds.b mds.imported=73 2024-03-11T20:03:51.299 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-11T20:03:51.299 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-11T20:03:51.299 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-11T20:03:51.299 DEBUG:tasks.check_counter:Getting stats from e 2024-03-11T20:03:51.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-11T20:03:52.034 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: pgmap v326: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.436 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.437 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.437 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.437 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.437 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.437 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:52 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: pgmap v326: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:52 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: pgmap v326: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:52.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:52 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.459 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.460 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.460 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.460 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:53 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.777 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:53 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi145 to 174.1M 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi145 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:53 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 4, 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 4, 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3024, 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 448, 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-03-11T20:03:53.827 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.001312930, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000034343, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000497222, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 3, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000680235, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000226745 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:53.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 242, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 231, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1569088, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 139136, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.047158620, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.014459962, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.036088696, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005644499, 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:53.829 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 230, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011169993, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000048565 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1569088, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 139136 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2, 2024-03-11T20:03:53.830 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 177, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 320, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 185952, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.012060918, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.008261453, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001471549, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 174, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.032006551, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000183945 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:53.831 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 185952 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-03-11T20:03:53.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:53.834 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:53.835 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148692, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 30, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:53.836 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:53.837 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 244, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1534964, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 244, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 244, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1534964, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:53.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:53.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:53.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:53.839 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-11T20:03:54.501 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:54 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T20:03:54.501 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:54 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T20:03:54.501 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:54 smithi122 ceph-mon[36397]: pgmap v327: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:03:54.570 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-03-11T20:03:54.570 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-03-11T20:03:54.570 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-03-11T20:03:54.571 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-03-11T20:03:54.571 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-03-11T20:03:54.571 DEBUG:tasks.check_counter:Getting stats from h 2024-03-11T20:03:54.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-11T20:03:54.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:54 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T20:03:54.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:54 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T20:03:54.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:54 smithi145 ceph-mon[36730]: pgmap v327: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:03:54.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:54 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-11T20:03:54.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:54 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-11T20:03:54.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:54 smithi107 ceph-mon[31712]: pgmap v327: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-11T20:03:54.833 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: pgmap v328: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T20:03:56.002 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T20:03:56.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:03:56.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:03:56.003 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:55 smithi122 ceph-mon[36397]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: pgmap v328: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:03:56.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:55 smithi145 ceph-mon[36730]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: pgmap v328: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: Adjusting osd_memory_target on smithi122 to 174.1M 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: Unable to set osd_memory_target on smithi122 to 182574694: error parsing value: Value '182574694' is below minimum 939524096 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T20:03:56.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:55 smithi107 ceph-mon[31712]: from='mgr.15324 ' entity='mgr.x' 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 1416, 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 1416, 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 21402211, 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 439590, 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 7, 2024-03-11T20:03:56.545 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.143014152, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.050296829, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.043064075, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.036606727, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1415, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.058534823, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000041367 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 1095, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 1073, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 14593732, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 397717, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.108918845, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.030784517, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.058436935, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.025439057, 2024-03-11T20:03:56.547 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1072, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.039122171, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000036494 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1639296, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 142320 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 391, 2024-03-11T20:03:56.548 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 565, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 8485553, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 582429, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 7, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.048338230, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.015871381, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.015168746, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.012407446, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 562, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.049016938, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000087218 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:56.549 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 461760 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-03-11T20:03:56.550 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 4393, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.167210410, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000038062 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-03-11T20:03:56.551 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 6436, 2024-03-11T20:03:56.552 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 4295, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 2141, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 300, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 2, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 2, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-03-11T20:03:56.553 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:03:56.554 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-03-11T20:03:56.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-03-11T20:03:56.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-03-11T20:03:56.557 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 12394, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 25, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 46139146, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 46300470, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 46300334, 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-03-11T20:03:56.558 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 12394 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1824, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 4215, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 2391, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 330, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 632, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 302, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 6436, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 7638, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 1202, 2024-03-11T20:03:56.559 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 75980, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215420 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.560 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.561 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.562 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-03-11T20:03:56.564 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.569 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.570 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 2, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 7, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 5, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 5, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-03-11T20:03:56.571 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 4732, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:56.572 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:56.573 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 304422, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 2479, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:56.574 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9816, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 205, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 10438720, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 837581, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:56.575 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 2621, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 26, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 4, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 2621, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2621, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 2.508940945, 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000957245 2024-03-11T20:03:56.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2621, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 2622 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 2621, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 2621, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 1619, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 999, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-03-11T20:03:56.577 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 2, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-03-11T20:03:56.578 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-03-11T20:03:56.579 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 106, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 16, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 8, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 8, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-03-11T20:03:56.580 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-03-11T20:03:56.581 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 2898, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 44203671, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 2898, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:56.582 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2898, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 44203671, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:56.583 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2621, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 42106176, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2621, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 42106176, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-03-11T20:03:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2621, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 2621, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2621, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 2621, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:03:56.585 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x560faae8e960": { 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-03-11T20:03:56.587 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:56.588 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-11T20:03:57.365 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-11T20:03:57.365 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-11T20:03:57.365 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-11T20:03:57.365 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-11T20:03:57.365 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T20:03:57.365 DEBUG:tasks.check_counter:Getting stats from k 2024-03-11T20:03:57.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-11T20:03:57.606 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T20:03:59.001 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:03:58 smithi122 ceph-mon[36397]: pgmap v329: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:59.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:03:58 smithi145 ceph-mon[36730]: pgmap v329: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:59.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:03:58 smithi107 ceph-mon[31712]: pgmap v329: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T20:03:59.250 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-11T20:03:59.250 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 250, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 237, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1573200, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 142080, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.049214756, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.015181136, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.036634122, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005923830, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 237, 2024-03-11T20:03:59.251 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.012954149, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000054658 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1573200, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 142080 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 181, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 320, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 186656, 2024-03-11T20:03:59.252 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.011376735, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.007859894, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001200694, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 178, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.036377472, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000204367 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:59.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 186656 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000257178, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000197193, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:03:59.254 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.255 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:03:59.256 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-03-11T20:03:59.257 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148732, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 31, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-03-11T20:03:59.258 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-03-11T20:03:59.259 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 248, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1535484, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 248, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 248, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1535484, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-11T20:03:59.260 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-11T20:03:59.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-11T20:03:59.944 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-11T20:03:59.944 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-11T20:03:59.944 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-11T20:03:59.944 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-11T20:03:59.944 DEBUG:tasks.check_counter:Getting stats from c 2024-03-11T20:03:59.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-11T20:04:00.193 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T20:04:01.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:00 smithi145 ceph-mon[36730]: pgmap v330: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:04:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:00 smithi107 ceph-mon[31712]: pgmap v330: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:04:01.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:00 smithi122 ceph-mon[36397]: pgmap v330: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1556, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1534, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 26327515, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 541799, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 7, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.196312800, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.055873547, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.086351260, 2024-03-11T20:04:02.136 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.049926425, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1533, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.069549016, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000045367 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1639680, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 143408 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:04:02.137 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 585, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 760, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 50483135, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 720220, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 10, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 7, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.094843416, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.021796153, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.046108925, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.019887435, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 758, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.053691531, 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000070833 2024-03-11T20:04:02.138 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 540400 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 834, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 835, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 63133156, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 259070, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-03-11T20:04:02.139 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.099194901, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.019072760, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.054244477, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.020899312, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 833, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.028335760, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000034016 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.140 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4385, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.123733675, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000028217 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.141 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.142 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 32130, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 49, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 32081, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 32060, 2024-03-11T20:04:02.143 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 2, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-03-11T20:04:02.144 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:04:02.145 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-03-11T20:04:02.146 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:04:02.147 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 52269, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-03-11T20:04:02.148 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 70, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 6, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 141726068, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 141721563, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 52269 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-03-11T20:04:02.149 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 11142, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 21536, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 10394, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 1067, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 1373, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 306, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 32130, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 33680, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 1550, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 113180, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215420 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-03-11T20:04:02.150 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-03-11T20:04:02.151 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.152 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.153 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.154 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-03-11T20:04:02.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.155 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.155 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.155 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-03-11T20:04:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.157 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.158 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.159 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.160 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.161 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 5, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 3, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 4, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3929, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:04:02.162 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-03-11T20:04:02.163 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-03-11T20:04:02.164 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 699923, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 12525, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-03-11T20:04:02.165 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9816, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 205, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 50795986, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 751894, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-03-11T20:04:02.166 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 2692, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 2692, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2692, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.326928109, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001235857 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2692, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2693 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.167 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 2692, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 2692, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1644, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 1046, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-03-11T20:04:02.168 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-03-11T20:04:02.169 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-03-11T20:04:02.170 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 106, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 16, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 11, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 11, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.171 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2971, 2024-03-11T20:04:02.172 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 139659310, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2971, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2971, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 139659310, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:02.173 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2692, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 137547916, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2692, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 137547916, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-03-11T20:04:02.174 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2692, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2692, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2692, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2692, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.175 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.176 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x564a38c1c960": { 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:02.177 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:02.178 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:02.178 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:02.178 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.178 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:02.178 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-03-11T20:04:02.880 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-11T20:04:02.881 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-11T20:04:02.881 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-11T20:04:02.881 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-11T20:04:02.881 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T20:04:02.881 DEBUG:tasks.check_counter:Getting stats from f 2024-03-11T20:04:02.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-11T20:04:03.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:02 smithi145 ceph-mon[36730]: pgmap v331: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:04:03.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:02 smithi107 ceph-mon[31712]: pgmap v331: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:04:03.133 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T20:04:03.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:02 smithi122 ceph-mon[36397]: pgmap v331: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-03-11T20:04:04.893 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:04 smithi145 ceph-mon[36730]: pgmap v332: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:05.001 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:04 smithi122 ceph-mon[36397]: pgmap v332: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 314, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 290, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 5846504, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 155060, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 2, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.056084863, 2024-03-11T20:04:05.017 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.016115840, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.041014477, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.006948524, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 290, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.016392986, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000056527 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1642976, 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 144480 2024-03-11T20:04:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1050, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1228, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 17813967, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 867174, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 9, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.099736148, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.031986369, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.030289581, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.029214683, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1224, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.067083248, 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000054806 2024-03-11T20:04:05.019 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 541552 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 1486, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 1488, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 25611893, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 462204, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 6, 2024-03-11T20:04:05.020 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.164208386, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.044079892, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.055881905, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.050753866, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1486, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.063508920, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000042738 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.021 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4318, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.118795581, 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000027511 2024-03-11T20:04:05.022 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 0, 2024-03-11T20:04:05.023 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 6450, 2024-03-11T20:04:05.024 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 14, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 6436, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 6430, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 1, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 1, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 0, 2024-03-11T20:04:05.025 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-03-11T20:04:05.026 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 0, 2024-03-11T20:04:05.027 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 0 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 0, 2024-03-11T20:04:05.028 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 0, 2024-03-11T20:04:05.029 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 18460, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 25, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 3, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-03-11T20:04:05.030 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 51103439, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 51097523, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 18460 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.031 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 2303, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 7999, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 5696, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 440, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 1004, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 564, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 6450, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 11147, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 4697, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 0, 2024-03-11T20:04:05.032 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 69400, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 215420 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.033 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.034 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-03-11T20:04:05.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.036 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.037 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.037 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.037 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-03-11T20:04:05.037 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:04 smithi107 ceph-mon[31712]: pgmap v332: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:05.037 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.038 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.039 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.042 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.043 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-03-11T20:04:05.044 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 3, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 3115, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:04:05.045 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:04:05.046 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:04:05.047 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 330670, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 2974, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:04:05.048 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9816, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 205, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 11123100, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 296429, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-03-11T20:04:05.049 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 2567, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 21, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 3, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 2567, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2567, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.416926289, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001331097 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.050 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2567, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2568 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 2567, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 2567, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 1573, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 992, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-03-11T20:04:05.051 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 1, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-03-11T20:04:05.052 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-03-11T20:04:05.053 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 106, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 16, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 9, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 9, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 0, 2024-03-11T20:04:05.054 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 2, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 0 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 0 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-03-11T20:04:05.055 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 2846, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 48999066, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 2846, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:05.056 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2846, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 48999066, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:05.057 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2567, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 46909140, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2567, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 46909140, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:05.058 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 2567, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 2567, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2567, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 2567, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:05.060 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x55bc82ee4960": { 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 0, 2024-03-11T20:04:05.061 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 0, 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:05.062 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-03-11T20:04:06.083 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-11T20:04:06.083 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-11T20:04:06.083 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-11T20:04:06.083 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-11T20:04:06.083 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T20:04:06.083 DEBUG:tasks.check_counter:Getting stats from i 2024-03-11T20:04:06.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-11T20:04:06.327 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T20:04:07.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:06 smithi145 ceph-mon[36730]: pgmap v333: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:07.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:06 smithi107 ceph-mon[31712]: pgmap v333: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:07.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:06 smithi122 ceph-mon[36397]: pgmap v333: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 30344, 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 45909, 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 14480355, 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 34768399, 2024-03-11T20:04:08.028 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 15, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 1.815757582, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.852567098, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 1.234705440, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.064968009, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 45906, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.873165528, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000019020 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 52172, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.007483402, 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000143 2024-03-11T20:04:08.029 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 9070448, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 19592864 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 2858, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 3034, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 736687, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 23014603, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 12, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 6, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.308441677, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.154663578, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.056585831, 2024-03-11T20:04:08.030 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.081699436, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3031, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.184685815, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000060932 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 8, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000977, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000122 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 4432, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 544400 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.031 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 9560, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 8888, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3291597, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 14082311, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 8, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 5, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.530061894, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.206911198, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.306323922, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.063058347, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 8888, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.270921228, 2024-03-11T20:04:08.032 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000030481 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 13747, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.001881106, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000136 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 5872 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 1, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-MDSRank": { 2024-03-11T20:04:08.033 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3831, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 11.105717867, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002898908 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "finisher-PurgeQueue": { 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "queue_len": 0, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "complete_latency": { 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3941, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.410018139, 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000104039 2024-03-11T20:04:08.034 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": { 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "request": 9406, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "reply": 9339, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "reply_latency": { 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 9339, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 340.536187242, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.036463881 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_reply": 0, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "forward": 67, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_complete": 12, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_fetch_keys": 0, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_commit": 38, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_split": 3, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_merge": 0, 2024-03-11T20:04:08.035 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes": 4560, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_top": 3, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_bottom": 100, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pin_tail": 4457, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_pinned": 4554, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_expired": 5082, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "inodes_with_caps": 1, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": 1, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "subtrees": 17, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse": 11969, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_hit": 10898, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_forward": 40, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_discover": 140, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_dir_fetch": 0, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_remote_ino": 3, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "traverse_lock": 10, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "load_cent": 0, 2024-03-11T20:04:08.036 INFO:teuthology.orchestra.run.smithi145.stdout: "q": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "exported": 289, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "exported_inodes": 4480, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "imported": 102, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "imported_inodes": 1202, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_dir_fetch": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_backtrace_fetch": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "openino_peer_discover": 5, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rfiles": 12118, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rbytes": 1788379046, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "root_rsnaps": 3, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_set_tag": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_inotable_repaired": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_inodes": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T20:04:08.037 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "scrub_file_inodes": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_inode_file_caps": 94, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_revoke": 2303, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_grant": 7772, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_trunc": 28, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps": 8265, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_caps_dirty": 4713, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_cap_release": 95, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "process_request_cap_release": 4552 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_cache": { 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_update_receipt": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_try_discover": 0, 2024-03-11T20:04:08.038 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_send_discover": 169, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "dir_handle_discover": 362, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays": 451, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_delayed": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_strays_enqueuing": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_created": 2350, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_enqueued": 1892, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_reintegrated": 20, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "strays_migrated": 4, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_processing": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_enqueued": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "num_recovering_prioritized": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_started": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "recovery_completed": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T20:04:08.039 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_exportdir": 289, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_flush": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragmentdir": 3, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_fragstats": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "ireq_inodestats": 0 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_log": { 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evlrg": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evadd": 18460, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evex": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evtrm": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "ev": 18460, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evexg": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "evexd": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "segadd": 25, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "segex": 0, 2024-03-11T20:04:08.040 INFO:teuthology.orchestra.run.smithi145.stdout: "segtrm": 0, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "seg": 25, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "segmjr": 3, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "segexg": 0, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "segexd": 0, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "expos": 4194304, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "wrpos": 51103439, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "rdpos": 0, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "jlat": { 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 2330, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 60.499920191, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.025965630 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "replayed": 0 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_mem": { 2024-03-11T20:04:08.041 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 2299, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "ino+": 7980, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "ino-": 5681, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dir": 474, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dir+": 1005, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dir-": 531, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dn": 4560, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dn+": 11559, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "dn-": 6999, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "cap": 1, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "cap+": 6111, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "cap-": 6110, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "rss": 117884, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "heap": 207228 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.042 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_server": { 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_client_request": 12309, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "dispatch_server_request": 659, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_request": 9406, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_client_session": 379, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "handle_peer_request": 1211, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_create_latency": { 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 4155, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 317.444946754, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.076400709 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getattr_latency": { 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 62, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.121882517, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001965847 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getfilelock_latency": { 2024-03-11T20:04:08.043 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "req_link_latency": { 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 38, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.504088766, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.013265493 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookup_latency": { 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1262, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.784381253, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002998717 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookuphash_latency": { 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.044 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupino_latency": { 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupname_latency": { 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupparent_latency": { 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lookupsnap_latency": { 2024-03-11T20:04:08.045 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.046 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.046 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.046 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.046 INFO:teuthology.orchestra.run.smithi145.stdout: "req_lssnap_latency": { 2024-03-11T20:04:08.046 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mkdir_latency": { 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 641, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 1.717797814, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.002679871 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mknod_latency": { 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 51, 2024-03-11T20:04:08.047 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.057296339, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001123457 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "req_mksnap_latency": { 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "req_open_latency": { 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 3, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.001507343, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000502447 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "req_readdir_latency": { 2024-03-11T20:04:08.048 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 479, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.322502985, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000673283 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rename_latency": { 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 106, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 2.456827790, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.023177620 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "req_renamesnap_latency": { 2024-03-11T20:04:08.049 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "req_snapdiff_latency": { 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmdir_latency": { 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 485, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 9.936347392, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.020487314 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.050 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmsnap_latency": { 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "req_rmxattr_latency": { 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setattr_latency": { 2024-03-11T20:04:08.051 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 36, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.546863304, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.015190647 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setdirlayout_latency": { 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setfilelock_latency": { 2024-03-11T20:04:08.052 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setlayout_latency": { 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "req_setxattr_latency": { 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.053 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "req_symlink_latency": { 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 98, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.096055773, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000980160 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "req_unlink_latency": { 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 1923, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 3.545689212, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.001843832 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_revoke_eviction": 0, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "cap_acquisition_throttle": 0, 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "req_getvxattr_latency": { 2024-03-11T20:04:08.054 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_sessions": { 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "session_count": 2, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "session_add": 4, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "session_remove": 2, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_open": 2, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "sessions_stale": 0, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "total_load": 3377, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "average_load": 1688, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "avg_session_uptime": 626, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "mdthresh_evicted": 0 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.055 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:04:08.056 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:04:08.057 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-03-11T20:04:08.058 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 4020058, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 3301, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 1232, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 14, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:04:08.059 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 9816, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 205, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 9992947, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 307480, 2024-03-11T20:04:08.060 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "objecter": { 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_active": 0, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_laggy": 0, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send": 6669, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_send_bytes": 48433438, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_resend": 28, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_reply": 6669, 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "op_latency": { 2024-03-11T20:04:08.061 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6669, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 38.108176455, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.005714226 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op_inflight": 0, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "oplen_avg": { 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 6669, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 6943 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op": 6669, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op_r": 143, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op_w": 6526, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op_rmw": 0, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "op_pg": 0, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_stat": 0, 2024-03-11T20:04:08.062 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_create": 10, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_read": 131, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_write": 2461, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writefull": 98, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_writesame": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_append": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_zero": 11, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_truncate": 3, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_delete": 3734, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_mapext": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_sparse_read": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_clonerange": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_getxattr": 12, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_setxattr": 20, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_cmpxattr": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_rmxattr": 0, 2024-03-11T20:04:08.063 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_resetxattrs": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_call": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_watch": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_notify": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_pgls_filter": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "osdop_other": 38, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_active": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_send": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_resend": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "linger_ping": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_active": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_send": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "poolop_resend": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_active": 0, 2024-03-11T20:04:08.064 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_send": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "poolstat_resend": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_active": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_send": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "statfs_resend": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "command_active": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "command_send": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "command_resend": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "map_epoch": 106, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "map_full": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "map_inc": 16, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_sessions": 12, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_open": 12, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_session_close": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_laggy": 0, 2024-03-11T20:04:08.065 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_wr": 301, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_rd": 24, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_del": 100 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "oft": { 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_objs": 1, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_kv_pairs": 0, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_updates": 0, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "omap_total_removes": 0 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osd-slow-ops": { 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "slow_ops_count": 1 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "purge_queue": { 2024-03-11T20:04:08.066 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_ops_high_water": 21, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executing_high_water": 14, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed_ops": 3722, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_executed": 1892, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "pq_item_in_journal": 0 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 42758, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 13776050, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 42758, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:08.067 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 42758, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 13776050, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_bytes": { 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:08.068 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6669, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 48677002, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6669, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 48677002, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-objecter_ops": { 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 1024, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 0, 2024-03-11T20:04:08.069 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 0, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 6669, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 6669, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 6669, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 6669, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle": { 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:08.070 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 1892, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 243756, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 1892, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 131, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 243756, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-write_buf_throttle-0x5596aaf846e0": { 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 3758096384, 2024-03-11T20:04:08.071 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 18460, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 46909135, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 18460, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 2330, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 46909135, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:08.072 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-03-11T20:04:08.878 INFO:tasks.check_counter:Daemon mds.i mds.exported=289 2024-03-11T20:04:08.879 INFO:tasks.check_counter:Daemon mds.i mds.imported=102 2024-03-11T20:04:08.879 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=3 2024-03-11T20:04:08.879 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-11T20:04:08.879 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-11T20:04:08.879 DEBUG:tasks.check_counter:Getting stats from l 2024-03-11T20:04:08.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-11T20:04:09.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:08 smithi107 ceph-mon[31712]: pgmap v334: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:04:09.117 INFO:teuthology.orchestra.run.smithi145.stderr:Inferring config /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T20:04:09.239 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:08 smithi145 ceph-mon[36730]: pgmap v334: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:04:09.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:08 smithi122 ceph-mon[36397]: pgmap v334: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-11T20:04:10.849 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:10 smithi145 ceph-mon[36730]: pgmap v335: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:10.849 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:10 smithi145 ceph-mon[36730]: osdmap e107: 12 total, 12 up, 12 in 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 0, 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 2, 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 0, 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 0, 2024-03-11T20:04:11.057 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 4, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 0, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.000946785, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.000153089, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:11.058 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 6, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 185, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 3344, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 187456, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 5, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 3, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.011539257, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.007812162, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.001488053, 2024-03-11T20:04:11.059 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 183, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.036692714, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000200506 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 187456 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.060 INFO:teuthology.orchestra.run.smithi145.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_messages": 250, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages": 239, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_bytes": 1571264, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_bytes": 144128, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_created_connections": 2, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_active_connections": 1, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_total_time": 0.045595499, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_send_time": 0.014752676, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_recv_time": 0.032486586, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_running_fast_dispatch_time": 0.005766949, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 238, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.012395191, 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000052080 2024-03-11T20:04:11.061 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_handle_ack_lat": { 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_recv_encrypted_bytes": 1571264, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "msgr_send_encrypted_bytes": 144128 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "cct": { 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "total_workers": 0, 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: "unhealthy_workers": 0 2024-03-11T20:04:11.062 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "mempool": { 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bloom_filter_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_alloc_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_data_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_other_items": 0, 2024-03-11T20:04:11.063 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_extent_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_blob_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_fsck_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_txc_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T20:04:11.064 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluestore_writing_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_reader_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "bluefs_file_writer_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_bytes": 148732, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_anon_items": 31, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "buffer_meta_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_bytes": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_mapbl_items": 0, 2024-03-11T20:04:11.065 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_pglog_items": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_items": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_mapping_items": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "pgmap_items": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_bytes": 928, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_co_items": 2, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_1_items": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_bytes": 0, 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: "unittest_2_items": 0 2024-03-11T20:04:11.066 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "val": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "max": 104857600, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_started": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get": 252, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_sum": 1536004, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_fail": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "get_or_fail_success": 252, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "take": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "take_sum": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "put": 252, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "put_sum": 1536004, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "wait": { 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "avgcount": 0, 2024-03-11T20:04:11.067 INFO:teuthology.orchestra.run.smithi145.stdout: "sum": 0.000000000, 2024-03-11T20:04:11.068 INFO:teuthology.orchestra.run.smithi145.stdout: "avgtime": 0.000000000 2024-03-11T20:04:11.068 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:11.068 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-03-11T20:04:11.068 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-03-11T20:04:11.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:10 smithi122 ceph-mon[36397]: pgmap v335: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:11.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:10 smithi122 ceph-mon[36397]: osdmap e107: 12 total, 12 up, 12 in 2024-03-11T20:04:11.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:10 smithi107 ceph-mon[31712]: pgmap v335: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-11T20:04:11.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:10 smithi107 ceph-mon[31712]: osdmap e107: 12 total, 12 up, 12 in 2024-03-11T20:04:12.056 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-11T20:04:12.056 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-11T20:04:12.056 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-11T20:04:12.056 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-11T20:04:12.056 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-11T20:04:12.056 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-11T20:04:12.067 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-11T20:04:12.067 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T20:04:12.067 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T20:04:12.141 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-11T20:04:12.142 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T20:04:12.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:04:12.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 -- 172.21.15.107:0/2147100196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 msgr2=0x7f969c108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 --2- 172.21.15.107:0/2147100196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c108180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f968c009900 tx=0x7f968c02bc30 comp rx=0 tx=0).stop 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 -- 172.21.15.107:0/2147100196 shutdown_connections 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 --2- 172.21.15.107:0/2147100196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 0x7f969c118600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 --2- 172.21.15.107:0/2147100196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 0x7f969c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 --2- 172.21.15.107:0/2147100196 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c108180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.513+0000 7f96a1c00640 1 -- 172.21.15.107:0/2147100196 >> 172.21.15.107:0/2147100196 conn(0x7f969c06c6d0 msgr2=0x7f969c06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:12.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.514+0000 7f96a1c00640 1 -- 172.21.15.107:0/2147100196 shutdown_connections 2024-03-11T20:04:12.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.515+0000 7f96a1c00640 1 -- 172.21.15.107:0/2147100196 wait complete. 2024-03-11T20:04:12.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.517+0000 7f96a1c00640 1 Processor -- start 2024-03-11T20:04:12.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.517+0000 7f96a1c00640 1 -- start start 2024-03-11T20:04:12.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.517+0000 7f96a1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 0x7f969c115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:12.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.517+0000 7f96a1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:12.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f96a1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 0x7f969c114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:12.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f96a1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c114bd0 con 0x7f969c1086c0 2024-03-11T20:04:12.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f96a1c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1b28c0 con 0x7f969c116210 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f96a1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1b2a80 con 0x7f969c10b890 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:44854/0 (socket says 172.21.15.107:44854) 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 -- 172.21.15.107:0/3924624672 learned_addr learned my addr 172.21.15.107:0/3924624672 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969b400640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 0x7f969c114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969aa00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 0x7f969c115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 -- 172.21.15.107:0/3924624672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 msgr2=0x7f969c114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 0x7f969c114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 -- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 msgr2=0x7f969c115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.518+0000 7f969a000640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 0x7f969c115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f969a000640 1 -- 172.21.15.107:0/3924624672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c009560 con 0x7f969c10b890 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f969a000640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f968800c920 tx=0x7f968800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688007c70 con 0x7f969c10b890 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c1b2d60 con 0x7f969c10b890 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9688007dd0 con 0x7f969c10b890 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c1b3260 con 0x7f969c10b890 2024-03-11T20:04:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.519+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688018860 con 0x7f969c10b890 2024-03-11T20:04:12.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.523+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c005290 con 0x7f969c10b890 2024-03-11T20:04:12.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.526+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f96880189c0 con 0x7f969c10b890 2024-03-11T20:04:12.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.528+0000 7f9693400640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 0x7f96800a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:12.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.528+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f96880cdcf0 con 0x7f969c10b890 2024-03-11T20:04:12.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.529+0000 7f969aa00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 0x7f96800a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:12.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.531+0000 7f969aa00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 0x7f96800a4120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f968c006010 tx=0x7f968c000bc0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:12.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.532+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96880d2070 con 0x7f969c10b890 2024-03-11T20:04:12.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.917+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f967c005810 con 0x7f969c10b890 2024-03-11T20:04:12.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.920+0000 7f9693400640 1 -- 172.21.15.107:0/3924624672 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25722 (secure 0 0 0) 0x7f9688096730 con 0x7f969c10b890 2024-03-11T20:04:12.913 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:04:12.914 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":107,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T20:04:09.784998+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/3157443745":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/2142594075":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1019473398":"2024-03-11T20:58:16.653084+0000","172.21.15.107:0/3959959887":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6801/1546798510":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/247446141":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/1546798510":"2024-03-12T19:53:25.384695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:04:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.923+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 msgr2=0x7f96800a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.923+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 0x7f96800a4120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f968c006010 tx=0x7f968c000bc0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.923+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 msgr2=0x7f969c113d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.923+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f968800c920 tx=0x7f968800cde0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 shutdown_connections 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f96800a1c60 0x7f96800a4120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f969c116210 0x7f969c114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f969c10b890 0x7f969c113d20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 --2- 172.21.15.107:0/3924624672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c1086c0 0x7f969c115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 >> 172.21.15.107:0/3924624672 conn(0x7f969c06c6d0 msgr2=0x7f969c1165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.924+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 shutdown_connections 2024-03-11T20:04:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:12.925+0000 7f96a1c00640 1 -- 172.21.15.107:0/3924624672 wait complete. 2024-03-11T20:04:12.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:44:40.865282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:49:53.416489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:49:54.674469+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T19:49:58.942750+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T20:04:12.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T20:04:13.025 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:12 smithi145 ceph-mon[36730]: pgmap v337: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:04:13.051 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:12 smithi107 ceph-mon[31712]: pgmap v337: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:04:13.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 -- 172.21.15.107:0/2410115110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c4107260 msgr2=0x7f29c4107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 --2- 172.21.15.107:0/2410115110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c4107260 0x7f29c4107640 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009ca0 tx=0x7f29b402c100 comp rx=0 tx=0).stop 2024-03-11T20:04:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 -- 172.21.15.107:0/2410115110 shutdown_connections 2024-03-11T20:04:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 --2- 172.21.15.107:0/2410115110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c40ffc10 0x7f29c410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 --2- 172.21.15.107:0/2410115110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ff250 0x7f29c40ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 --2- 172.21.15.107:0/2410115110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c4107260 0x7f29c4107640 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.256+0000 7f29ca400640 1 -- 172.21.15.107:0/2410115110 >> 172.21.15.107:0/2410115110 conn(0x7f29c40761d0 msgr2=0x7f29c40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:13.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.257+0000 7f29ca400640 1 -- 172.21.15.107:0/2410115110 shutdown_connections 2024-03-11T20:04:13.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.257+0000 7f29ca400640 1 -- 172.21.15.107:0/2410115110 wait complete. 2024-03-11T20:04:13.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:12 smithi122 ceph-mon[36397]: pgmap v337: 129 pgs: 15 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 71 active+clean; 202 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.258+0000 7f29ca400640 1 Processor -- start 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.258+0000 7f29ca400640 1 -- start start 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.258+0000 7f29ca400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.258+0000 7f29ca400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 0x7f29c4110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29ca400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 0x7f29c4110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29ca400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41a9fe0 con 0x7f29c4107260 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29ca400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41aa120 con 0x7f29c40ff250 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29ca400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41aa260 con 0x7f29c40ffc10 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:48000/0 (socket says 172.21.15.107:48000) 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 -- 172.21.15.107:0/2447243871 learned_addr learned my addr 172.21.15.107:0/2447243871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3e00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 0x7f29c4110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c2a00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 0x7f29c4110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 -- 172.21.15.107:0/2447243871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 msgr2=0x7f29c4110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 0x7f29c4110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.259+0000 7f29c3400640 1 -- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 msgr2=0x7f29c4110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.260+0000 7f29c3400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 0x7f29c4110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.260+0000 7f29c3400640 1 -- 172.21.15.107:0/2447243871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b4009900 con 0x7f29c40ff250 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.260+0000 7f29c2a00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 0x7f29c4110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.260+0000 7f29c3e00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 0x7f29c4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.260+0000 7f29c3400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f80 tx=0x7f29b40325f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.261+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4034850 con 0x7f29c40ff250 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.261+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29b4003e50 con 0x7f29c40ff250 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.261+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4040450 con 0x7f29c40ff250 2024-03-11T20:04:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.262+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c41aa4b0 con 0x7f29c40ff250 2024-03-11T20:04:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.263+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41aa910 con 0x7f29c40ff250 2024-03-11T20:04:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.266+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c4100670 con 0x7f29c40ff250 2024-03-11T20:04:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.266+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f29b400a040 con 0x7f29c40ff250 2024-03-11T20:04:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.269+0000 7f29bbe00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 0x7f29a80a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.270+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f29b40efce0 con 0x7f29c40ff250 2024-03-11T20:04:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.271+0000 7f29c2a00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 0x7f29a80a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.272+0000 7f29c2a00640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 0x7f29a80a41b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29c41117a0 tx=0x7f29b0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:13.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.274+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29b40b88b0 con 0x7f29c40ff250 2024-03-11T20:04:13.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.626+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29c4101f20 con 0x7f29c40ff250 2024-03-11T20:04:13.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.627+0000 7f29bbe00640 1 -- 172.21.15.107:0/2447243871 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29b4044020 con 0x7f29c40ff250 2024-03-11T20:04:13.621 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-11T20:04:13.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 msgr2=0x7f29a80a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 0x7f29a80a41b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29c41117a0 tx=0x7f29b0005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:13.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 msgr2=0x7f29c410ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29b4000f80 tx=0x7f29b40325f0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 shutdown_connections 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f29a80a1cf0 0x7f29a80a41b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c4107260 0x7f29c4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29c40ffc10 0x7f29c4110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 --2- 172.21.15.107:0/2447243871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29c40ff250 0x7f29c410ffe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 >> 172.21.15.107:0/2447243871 conn(0x7f29c40761d0 msgr2=0x7f29c40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.632+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 shutdown_connections 2024-03-11T20:04:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.633+0000 7f29ca400640 1 -- 172.21.15.107:0/2447243871 wait complete. 2024-03-11T20:04:13.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T20:04:13.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.944+0000 7f9c05a00640 1 -- 172.21.15.107:0/2544349267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c0010d9b0 msgr2=0x7f9c0010fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.944+0000 7f9c05a00640 1 --2- 172.21.15.107:0/2544349267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010fdc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009ca0 tx=0x7f9bf802c180 comp rx=0 tx=0).stop 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.945+0000 7f9c05a00640 1 -- 172.21.15.107:0/2544349267 shutdown_connections 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.945+0000 7f9c05a00640 1 --2- 172.21.15.107:0/2544349267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010fdc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.945+0000 7f9c05a00640 1 --2- 172.21.15.107:0/2544349267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00073e70 0x7f9c0010d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.945+0000 7f9c05a00640 1 --2- 172.21.15.107:0/2544349267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c00073550 0x7f9c00073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.945+0000 7f9c05a00640 1 -- 172.21.15.107:0/2544349267 >> 172.21.15.107:0/2544349267 conn(0x7f9c00078580 msgr2=0x7f9c00078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.946+0000 7f9c05a00640 1 -- 172.21.15.107:0/2544349267 shutdown_connections 2024-03-11T20:04:13.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.946+0000 7f9c05a00640 1 -- 172.21.15.107:0/2544349267 wait complete. 2024-03-11T20:04:13.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.947+0000 7f9c05a00640 1 Processor -- start 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 -- start start 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00073550 0x7f9c0010d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0010af40 con 0x7f9c00073550 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0010b0b0 con 0x7f9c00073e70 2024-03-11T20:04:13.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.948+0000 7f9c05a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c001aa310 con 0x7f9c0010d9b0 2024-03-11T20:04:13.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:48024/0 (socket says 172.21.15.107:48024) 2024-03-11T20:04:13.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 -- 172.21.15.107:0/3534350781 learned_addr learned my addr 172.21.15.107:0/3534350781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:13.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bff400640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 -- 172.21.15.107:0/3534350781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c0010d9b0 msgr2=0x7f9c0010a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 -- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00073550 msgr2=0x7f9c0010d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:13.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00073550 0x7f9c0010d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:13.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.949+0000 7f9bfe000640 1 -- 172.21.15.107:0/3534350781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8009900 con 0x7f9c00073e70 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9bfe000640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec009700 con 0x7f9c00073e70 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bec00cd80 con 0x7f9c00073e70 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0188e0 con 0x7f9c00073e70 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c001aa510 con 0x7f9c00073e70 2024-03-11T20:04:13.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.950+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c001aa940 con 0x7f9c00073e70 2024-03-11T20:04:13.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.952+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c00074c00 con 0x7f9c00073e70 2024-03-11T20:04:13.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.957+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f9bec010b10 con 0x7f9c00073e70 2024-03-11T20:04:13.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.959+0000 7f9bf7400640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 0x7f9be40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:13.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.960+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f9bec014070 con 0x7f9c00073e70 2024-03-11T20:04:13.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.962+0000 7f9bfea00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 0x7f9be40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:13.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.963+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bec096a00 con 0x7f9c00073e70 2024-03-11T20:04:13.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:13.963+0000 7f9bfea00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 0x7f9be40a40e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf0005c70 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:14.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:13 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3924624672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:14.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:13 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2447243871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:04:14.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:13 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3924624672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:14.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:13 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2447243871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:04:14.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:13 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3924624672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:14.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:13 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2447243871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T20:04:14.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.306+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c001134e0 con 0x7f9c00073e70 2024-03-11T20:04:14.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.306+0000 7f9bf7400640 1 -- 172.21.15.107:0/3534350781 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9bec09a0c0 con 0x7f9c00073e70 2024-03-11T20:04:14.299 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T20:04:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 msgr2=0x7f9be40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 0x7f9be40a40e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf0005c70 comp rx=0 tx=0).stop 2024-03-11T20:04:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 msgr2=0x7f9c0010a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 shutdown_connections 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f9be40a1c20 0x7f9be40a40e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c00073e70 0x7f9c0010a270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c0010d9b0 0x7f9c0010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 --2- 172.21.15.107:0/3534350781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c00073550 0x7f9c0010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.309+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 >> 172.21.15.107:0/3534350781 conn(0x7f9c00078580 msgr2=0x7f9c000ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.310+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 shutdown_connections 2024-03-11T20:04:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.310+0000 7f9c05a00640 1 -- 172.21.15.107:0/3534350781 wait complete. 2024-03-11T20:04:14.329 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 -- 172.21.15.107:0/61592934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc11a960 msgr2=0x7fa9cc11cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/61592934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc11a960 0x7fa9cc11cd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009ca0 tx=0x7fa9c402c100 comp rx=0 tx=0).stop 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 -- 172.21.15.107:0/61592934 shutdown_connections 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/61592934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/61592934 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc11a960 0x7fa9cc11cd50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/61592934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.606+0000 7fa9d2e00640 1 -- 172.21.15.107:0/61592934 >> 172.21.15.107:0/61592934 conn(0x7fa9cc06c260 msgr2=0x7fa9cc06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.608+0000 7fa9d2e00640 1 -- 172.21.15.107:0/61592934 shutdown_connections 2024-03-11T20:04:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.608+0000 7fa9d2e00640 1 -- 172.21.15.107:0/61592934 wait complete. 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 Processor -- start 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 -- start start 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc1a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 0x7fa9cc1aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1ab420 con 0x7fa9cc11a960 2024-03-11T20:04:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1ab590 con 0x7fa9cc10d9d0 2024-03-11T20:04:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.610+0000 7fa9d2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1ab750 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc1a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:48032/0 (socket says 172.21.15.107:48032) 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cbe00640 1 -- 172.21.15.107:0/2069353257 learned_addr learned my addr 172.21.15.107:0/2069353257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc1a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9d0e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 0x7fa9cc1aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 -- 172.21.15.107:0/2069353257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 msgr2=0x7fa9cc1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 -- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 msgr2=0x7fa9cc1aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 0x7fa9cc1aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9cb400640 1 -- 172.21.15.107:0/2069353257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc009490 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.611+0000 7fa9d0e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 0x7fa9cc1aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T20:04:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.612+0000 7fa9cb400640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc1a6aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9f0 tx=0x7fa9bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.613+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc00cd80 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.613+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9bc0035f0 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.613+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc010690 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.613+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c4009900 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.613+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc1b30d0 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.615+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc067f70 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.618+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fa9bc003af0 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.620+0000 7fa9c8c00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 0x7fa9b00a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.620+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7fa9bc014070 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.620+0000 7fa9cbe00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 0x7fa9b00a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.622+0000 7fa9cbe00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 0x7fa9b00a4120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009c10 tx=0x7fa9b8005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.628+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9bc096ae0 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.991+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9cc120090 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.992+0000 7fa9c8c00640 1 -- 172.21.15.107:0/2069353257 <== mon.1 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9bc09a1a0 con 0x7fa9cc10f9a0 2024-03-11T20:04:14.985 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 msgr2=0x7fa9b00a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 0x7fa9b00a4120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009c10 tx=0x7fa9b8005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 msgr2=0x7fa9cc1a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc1a6aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00e9f0 tx=0x7fa9bc00eeb0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 shutdown_connections 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fa9b00a1c60 0x7fa9b00a4120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9cc11a960 0x7fa9cc1aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9cc10f9a0 0x7fa9cc1a6aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 --2- 172.21.15.107:0/2069353257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9cc10d9d0 0x7fa9cc1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.996+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 >> 172.21.15.107:0/2069353257 conn(0x7fa9cc06c260 msgr2=0x7fa9cc11c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.997+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 shutdown_connections 2024-03-11T20:04:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:14.997+0000 7fa9d2e00640 1 -- 172.21.15.107:0/2069353257 wait complete. 2024-03-11T20:04:15.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T20:04:15.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:14 smithi107 ceph-mon[31712]: pgmap v338: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:15.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:14 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3534350781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:04:15.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:14 smithi122 ceph-mon[36397]: pgmap v338: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:15.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:14 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3534350781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:04:15.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:14 smithi145 ceph-mon[36730]: pgmap v338: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:15.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:14 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3534350781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3533496403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a40ffd90 msgr2=0x7fc0a4100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3533496403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a40ffd90 0x7fc0a4100170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc094009900 tx=0x7fc09402bc30 comp rx=0 tx=0).stop 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3533496403 shutdown_connections 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3533496403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a410d940 0x7fc0a410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3533496403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4100740 0x7fc0a410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3533496403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a40ffd90 0x7fc0a4100170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3533496403 >> 172.21.15.107:0/3533496403 conn(0x7fc0a40fb9d0 msgr2=0x7fc0a40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3533496403 shutdown_connections 2024-03-11T20:04:15.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.288+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3533496403 wait complete. 2024-03-11T20:04:15.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.289+0000 7fc0a3e00640 1 Processor -- start 2024-03-11T20:04:15.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.289+0000 7fc0a3e00640 1 -- start start 2024-03-11T20:04:15.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 0x7fc0a419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4100740 0x7fc0a419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4199e20 con 0x7fc0a4100740 2024-03-11T20:04:15.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4199f90 con 0x7fc0a410d940 2024-03-11T20:04:15.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.290+0000 7fc0a3e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419a150 con 0x7fc0a40ffd90 2024-03-11T20:04:15.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 0x7fc0a419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:37384/0 (socket says 172.21.15.107:37384) 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 -- 172.21.15.107:0/3714558602 learned_addr learned my addr 172.21.15.107:0/3714558602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 0x7fc0a419f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33540/0 (socket says 172.21.15.107:33540) 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 -- 172.21.15.107:0/3714558602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 msgr2=0x7fc0a419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 0x7fc0a419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 -- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4100740 msgr2=0x7fc0a419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4100740 0x7fc0a419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.291+0000 7fc0a3400640 1 -- 172.21.15.107:0/3714558602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094009560 con 0x7fc0a410d940 2024-03-11T20:04:15.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.292+0000 7fc0a3400640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9d0 tx=0x7fc09c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:15.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.292+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00cd80 con 0x7fc0a410d940 2024-03-11T20:04:15.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.292+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc09c005020 con 0x7fc0a410d940 2024-03-11T20:04:15.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.292+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0116d0 con 0x7fc0a410d940 2024-03-11T20:04:15.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.293+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a419a430 con 0x7fc0a410d940 2024-03-11T20:04:15.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.293+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a41a6550 con 0x7fc0a410d940 2024-03-11T20:04:15.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.294+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a4068b60 con 0x7fc0a410d940 2024-03-11T20:04:15.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.297+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fc09c005500 con 0x7fc0a410d940 2024-03-11T20:04:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.299+0000 7fc09b400640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 0x7fc0880a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.299+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7fc09c0cde30 con 0x7fc0a410d940 2024-03-11T20:04:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.300+0000 7fc0a2a00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 0x7fc0880a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.301+0000 7fc0a2a00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 0x7fc0880a4160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc094005ab0 tx=0x7fc094005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:15.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.306+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc09c096810 con 0x7fc0a410d940 2024-03-11T20:04:15.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.628+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc0a4108ad0 con 0x7fc0a410d940 2024-03-11T20:04:15.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.629+0000 7fc09b400640 1 -- 172.21.15.107:0/3714558602 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc09c01f020 con 0x7fc0a410d940 2024-03-11T20:04:15.622 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-11T20:04:15.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 msgr2=0x7fc0880a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 0x7fc0880a4160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc094005ab0 tx=0x7fc094005a20 comp rx=0 tx=0).stop 2024-03-11T20:04:15.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 msgr2=0x7fc0a4199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9d0 tx=0x7fc09c00ee90 comp rx=0 tx=0).stop 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 shutdown_connections 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fc0880a1ca0 0x7fc0880a4160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0a410d940 0x7fc0a4199690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a4100740 0x7fc0a419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 --2- 172.21.15.107:0/3714558602 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0a40ffd90 0x7fc0a419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 >> 172.21.15.107:0/3714558602 conn(0x7fc0a40fb9d0 msgr2=0x7fc0a40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.632+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 shutdown_connections 2024-03-11T20:04:15.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.633+0000 7fc0a3e00640 1 -- 172.21.15.107:0/3714558602 wait complete. 2024-03-11T20:04:15.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T20:04:15.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 -- 172.21.15.107:0/2522150325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7318103470 msgr2=0x7f7318103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 --2- 172.21.15.107:0/2522150325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7318103470 0x7f7318103850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7308009900 tx=0x7f730802bc30 comp rx=0 tx=0).stop 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 -- 172.21.15.107:0/2522150325 shutdown_connections 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 --2- 172.21.15.107:0/2522150325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 0x7f7318110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 --2- 172.21.15.107:0/2522150325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103e20 0x7f7318107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 --2- 172.21.15.107:0/2522150325 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7318103470 0x7f7318103850 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 -- 172.21.15.107:0/2522150325 >> 172.21.15.107:0/2522150325 conn(0x7f73180fecb0 msgr2=0x7f73181010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 -- 172.21.15.107:0/2522150325 shutdown_connections 2024-03-11T20:04:15.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.869+0000 7f731d200640 1 -- 172.21.15.107:0/2522150325 wait complete. 2024-03-11T20:04:15.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.870+0000 7f731d200640 1 Processor -- start 2024-03-11T20:04:15.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.870+0000 7f731d200640 1 -- start start 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.870+0000 7f731d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.870+0000 7f731d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 0x7f731819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.870+0000 7f731d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73181996f0 0x7f7318199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f731d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731819a1c0 con 0x7f7318103470 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f731d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731819a330 con 0x7f7318108540 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f731d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731819a4f0 con 0x7f73181996f0 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39784/0 (socket says 172.21.15.107:39784) 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 -- 172.21.15.107:0/3285034593 learned_addr learned my addr 172.21.15.107:0/3285034593 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317e00640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73181996f0 0x7f7318199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f730fe00640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 0x7f731819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 -- 172.21.15.107:0/3285034593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73181996f0 msgr2=0x7f7318199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73181996f0 0x7f7318199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 -- 172.21.15.107:0/3285034593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 msgr2=0x7f731819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:15.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 0x7f731819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 -- 172.21.15.107:0/3285034593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009490 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.871+0000 7f7317400640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7308000c00 tx=0x7f730802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.872+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730802dea0 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.872+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7308004490 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.872+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7308009560 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.872+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308003c90 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.872+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73181a67d0 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.873+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7318104940 con 0x7f7318103470 2024-03-11T20:04:15.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.875+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f7308004df0 con 0x7f7318103470 2024-03-11T20:04:15.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.876+0000 7f7314c00640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 0x7f72fc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:15.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.876+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f73080f0190 con 0x7f7318103470 2024-03-11T20:04:15.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.876+0000 7f730fe00640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 0x7f72fc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:15.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.877+0000 7f730fe00640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 0x7f72fc0a40e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f731819af40 tx=0x7f7300005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:15.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:15.880+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73080b8c80 con 0x7f7318103470 2024-03-11T20:04:16.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2069353257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:15 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3714558602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2069353257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:15 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3714558602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.080+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f73181a63d0 con 0x7f7318103470 2024-03-11T20:04:16.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.081+0000 7f7314c00640 1 -- 172.21.15.107:0/3285034593 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v107) v1 ==== 81+0+1016 (secure 0 0 0) 0x7f7308043410 con 0x7f7318103470 2024-03-11T20:04:16.074 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2887016959 2024-03-12T19:41:54.175068+0000 2024-03-11T20:04:16.074 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3157443745 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/927061211 2024-03-12T19:41:19.200454+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1620686143 2024-03-12T19:41:54.175068+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2142594075 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/531077471 2024-03-12T19:41:19.200454+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/865925559 2024-03-12T19:41:19.200454+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3464148109 2024-03-12T19:41:54.175068+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3318828522 2024-03-12T19:41:19.200454+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1356730303 2024-03-12T19:41:54.175068+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1019473398 2024-03-11T20:58:16.653084+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3959959887 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1546798510 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/247446141 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.075 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1546798510 2024-03-12T19:53:25.384695+0000 2024-03-11T20:04:16.076 INFO:teuthology.orchestra.run.smithi107.stderr:listed 17 entries 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 msgr2=0x7f72fc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 0x7f72fc0a40e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f731819af40 tx=0x7f7300005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 msgr2=0x7f731819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7308000c00 tx=0x7f730802c720 comp rx=0 tx=0).stop 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 shutdown_connections 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f72fc0a1c20 0x7f72fc0a40e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7318108540 0x7f731819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73181996f0 0x7f7318199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 --2- 172.21.15.107:0/3285034593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7318103470 0x7f731819f630 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 >> 172.21.15.107:0/3285034593 conn(0x7f73180fecb0 msgr2=0x7f7318100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 shutdown_connections 2024-03-11T20:04:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.084+0000 7f731d200640 1 -- 172.21.15.107:0/3285034593 wait complete. 2024-03-11T20:04:16.093 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-11T20:04:16.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:04:16.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-11T20:04:16.222 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T20:04:16.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:04:16.222 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-11T20:04:16.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2069353257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:15 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3714558602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-11T20:04:16.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:04:16.295 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-11T20:04:16.333 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-11T20:04:16.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:04:16.334 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T20:04:16.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:04:16.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-03-11T20:04:16.549 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-11T20:04:16.560 INFO:tasks.cephadm:Teardown begin 2024-03-11T20:04:16.560 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-11T20:04:16.560 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.742+0000 7f5b17000640 1 -- 172.21.15.107:0/3912457102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b10103f30 msgr2=0x7f5b10107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.742+0000 7f5b17000640 1 --2- 172.21.15.107:0/3912457102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b10103f30 0x7f5b10107f70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009930 tx=0x7f5afc02bc20 comp rx=0 tx=0).stop 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 -- 172.21.15.107:0/3912457102 shutdown_connections 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 --2- 172.21.15.107:0/3912457102 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b10110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 --2- 172.21.15.107:0/3912457102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b10103f30 0x7f5b10107f70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 --2- 172.21.15.107:0/3912457102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 0x7f5b10103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 -- 172.21.15.107:0/3912457102 >> 172.21.15.107:0/3912457102 conn(0x7f5b100fede0 msgr2=0x7f5b10101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 -- 172.21.15.107:0/3912457102 shutdown_connections 2024-03-11T20:04:16.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.743+0000 7f5b17000640 1 -- 172.21.15.107:0/3912457102 wait complete. 2024-03-11T20:04:16.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 Processor -- start 2024-03-11T20:04:16.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 -- start start 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 0x7f5b1019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b1019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 0x7f5b10199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1019a2e0 con 0x7f5b101998f0 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1019a450 con 0x7f5b10103580 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b17000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1019a610 con 0x7f5b10108650 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b0f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b1019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b15000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 0x7f5b10199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b0f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b1019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.107:33580/0 (socket says 172.21.15.107:33580) 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.744+0000 7f5b0f400640 1 -- 172.21.15.107:0/184927130 learned_addr learned my addr 172.21.15.107:0/184927130 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b0fe00640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 0x7f5b1019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 -- 172.21.15.107:0/184927130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 msgr2=0x7f5b1019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b1019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 -- 172.21.15.107:0/184927130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 msgr2=0x7f5b1019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 0x7f5b1019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 -- 172.21.15.107:0/184927130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009560 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b15000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 0x7f5b10199da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009fd0 tx=0x7f5b0800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800ce20 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b08005020 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b101a65b0 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08010690 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.745+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b101a6a90 con 0x7f5b101998f0 2024-03-11T20:04:16.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.747+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b100683c0 con 0x7f5b101998f0 2024-03-11T20:04:16.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.748+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f5b08005500 con 0x7f5b101998f0 2024-03-11T20:04:16.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.750+0000 7f5b0cc00640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:16.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.750+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f5b08014070 con 0x7f5b101998f0 2024-03-11T20:04:16.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.750+0000 7f5b0fe00640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:16.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.751+0000 7f5b0fe00640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 0x7f5af40a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b10102510 tx=0x7f5b00005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:16.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.754+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b08096d60 con 0x7f5b101998f0 2024-03-11T20:04:16.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.977+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b1019add0 con 0x7f5b101998f0 2024-03-11T20:04:16.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.978+0000 7f5b0cc00640 1 -- 172.21.15.107:0/184927130 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f5b08005190 con 0x7f5b101998f0 2024-03-11T20:04:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:04:16.972 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T20:01:38.218441+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T20:04:16.972 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-03-11T20:04:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 msgr2=0x7f5af40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 0x7f5af40a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5b10102510 tx=0x7f5b00005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 msgr2=0x7f5b10199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 0x7f5b10199da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009fd0 tx=0x7f5b0800ef10 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 shutdown_connections 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b101998f0 0x7f5b10199da0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5b10108650 0x7f5b1019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 --2- 172.21.15.107:0/184927130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b10103580 0x7f5b1019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 >> 172.21.15.107:0/184927130 conn(0x7f5b100fede0 msgr2=0x7f5b10100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.981+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 shutdown_connections 2024-03-11T20:04:16.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:16.982+0000 7f5b17000640 1 -- 172.21.15.107:0/184927130 wait complete. 2024-03-11T20:04:16.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T20:04:17.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:16 smithi145 ceph-mon[36730]: pgmap v339: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:17.026 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:16 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3285034593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T20:04:17.070 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:16 smithi107 ceph-mon[31712]: pgmap v339: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:17.070 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:16 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3285034593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 -- 172.21.15.107:0/4188122340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416410d740 msgr2=0x7f416410db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 --2- 172.21.15.107:0/4188122340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416410d740 0x7f416410db20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4154009860 tx=0x7f415402bb70 comp rx=0 tx=0).stop 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 -- 172.21.15.107:0/4188122340 shutdown_connections 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 --2- 172.21.15.107:0/4188122340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f4164114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 --2- 172.21.15.107:0/4188122340 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4164107730 0x7f4164107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 --2- 172.21.15.107:0/4188122340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416410d740 0x7f416410db20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 -- 172.21.15.107:0/4188122340 >> 172.21.15.107:0/4188122340 conn(0x7f41640756d0 msgr2=0x7f4164077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 -- 172.21.15.107:0/4188122340 shutdown_connections 2024-03-11T20:04:17.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.205+0000 7f416ae00640 1 -- 172.21.15.107:0/4188122340 wait complete. 2024-03-11T20:04:17.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.206+0000 7f416ae00640 1 Processor -- start 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.206+0000 7f416ae00640 1 -- start start 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.206+0000 7f416ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 0x7f416419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.206+0000 7f416ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.206+0000 7f416ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416410d740 0x7f41641a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f416ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641a2ca0 con 0x7f41641080f0 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f416ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641a2e10 con 0x7f4164107730 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f416ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641a2fd0 con 0x7f416410d740 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 0x7f416419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39804/0 (socket says 172.21.15.107:39804) 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163400640 1 -- 172.21.15.107:0/3857434588 learned_addr learned my addr 172.21.15.107:0/3857434588 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:17.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4168e00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416410d740 0x7f41641a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 -- 172.21.15.107:0/3857434588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416410d740 msgr2=0x7f41641a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416410d740 0x7f41641a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 -- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 msgr2=0x7f416419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 -- 172.21.15.107:0/3857434588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154009490 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163400640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.207+0000 7f4163e00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 0x7f416419de30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f415402c0b0 tx=0x7f41540045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.208+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540349d0 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.208+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f415402c9f0 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.208+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41641a3250 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.208+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540035f0 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.208+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41641aab30 con 0x7f4164107730 2024-03-11T20:04:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.209+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4164108e70 con 0x7f4164107730 2024-03-11T20:04:17.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.211+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f415403b040 con 0x7f4164107730 2024-03-11T20:04:17.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.212+0000 7f4160c00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 0x7f41480a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.212+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f41540ef950 con 0x7f4164107730 2024-03-11T20:04:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.212+0000 7f4163400640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 0x7f41480a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.213+0000 7f4163400640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 0x7f41480a40e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f416410cda0 tx=0x7f4150005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:17.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.214+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41540b85e0 con 0x7f4164107730 2024-03-11T20:04:17.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:16 smithi122 ceph-mon[36397]: pgmap v339: 129 pgs: 19 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 56 active+clean; 200 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-11T20:04:17.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:16 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3285034593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T20:04:17.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.435+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41640683c0 con 0x7f4164107730 2024-03-11T20:04:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.436+0000 7f4160c00640 1 -- 172.21.15.107:0/3857434588 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f4154041090 con 0x7f4164107730 2024-03-11T20:04:17.429 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:04:17.429 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3292957465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3292957465},{"type":"v1","addr":"172.21.15.107:6837","nonce":3292957465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/988917516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":988917516},{"type":"v1","addr":"172.21.15.107:6839","nonce":988917516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6833/1918081210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":1918081210},{"type":"v1","addr":"172.21.15.145:6833","nonce":1918081210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1820097321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1820097321},{"type":"v1","addr":"172.21.15.122:6833","nonce":1820097321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24478,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/1132278897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1132278897},{"type":"v1","addr":"172.21.15.107:6835","nonce":1132278897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1916658439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1916658439},{"type":"v1","addr":"172.21.15.122:6835","nonce":1916658439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:49:55.178113+0000","modified":"2024-03-11T20:01:38.218441+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24590,"mds_1":24583,"mds_2":24547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6835/3941680931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3941680931},{"type":"v1","addr":"172.21.15.145:6835","nonce":3941680931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.145:6837/2305866182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":2305866182},{"type":"v1","addr":"172.21.15.145:6837","nonce":2305866182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/547863696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":547863696},{"type":"v1","addr":"172.21.15.122:6837","nonce":547863696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.145:6839/322644420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6838","nonce":322644420},{"type":"v1","addr":"172.21.15.145:6839","nonce":322644420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24583":{"gid":24583,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.107:6841/981878935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":981878935},{"type":"v1","addr":"172.21.15.107:6841","nonce":981878935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24590":{"gid":24590,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6839/1167549353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1167549353},{"type":"v1","addr":"172.21.15.122:6839","nonce":1167549353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T20:04:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-03-11T20:04:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.438+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 msgr2=0x7f41480a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.438+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 0x7f41480a40e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f416410cda0 tx=0x7f4150005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.438+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 msgr2=0x7f416419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.438+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 0x7f416419de30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f415402c0b0 tx=0x7f41540045b0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 shutdown_connections 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f41480a1c20 0x7f41480a40e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f416410d740 0x7f41641a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41641080f0 0x7f416419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 --2- 172.21.15.107:0/3857434588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4164107730 0x7f416419de30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 >> 172.21.15.107:0/3857434588 conn(0x7f41640756d0 msgr2=0x7f4164075ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 shutdown_connections 2024-03-11T20:04:17.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.439+0000 7f416ae00640 1 -- 172.21.15.107:0/3857434588 wait complete. 2024-03-11T20:04:17.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.669+0000 7f1f03200640 1 -- 172.21.15.107:0/237693991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 msgr2=0x7f1f041037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.669+0000 7f1f03200640 1 --2- 172.21.15.107:0/237693991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f041037a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009930 tx=0x7f1ef402bc20 comp rx=0 tx=0).stop 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 -- 172.21.15.107:0/237693991 shutdown_connections 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 --2- 172.21.15.107:0/237693991 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04108490 0x7f1f04110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 --2- 172.21.15.107:0/237693991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04103d70 0x7f1f04107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 --2- 172.21.15.107:0/237693991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f041037a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 -- 172.21.15.107:0/237693991 >> 172.21.15.107:0/237693991 conn(0x7f1f040fec20 msgr2=0x7f1f04101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 -- 172.21.15.107:0/237693991 shutdown_connections 2024-03-11T20:04:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.670+0000 7f1f03200640 1 -- 172.21.15.107:0/237693991 wait complete. 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.671+0000 7f1f03200640 1 Processor -- start 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.671+0000 7f1f03200640 1 -- start start 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.671+0000 7f1f03200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.671+0000 7f1f03200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04108490 0x7f1f0419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f03200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 0x7f1f04199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f03200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0419a110 con 0x7f1f04108490 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f03200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0419a280 con 0x7f1f041033c0 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f03200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0419a440 con 0x7f1f04199640 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:37446/0 (socket says 172.21.15.107:37446) 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 -- 172.21.15.107:0/862806916 learned_addr learned my addr 172.21.15.107:0/862806916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01400640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04108490 0x7f1f0419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f02800640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 0x7f1f04199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 -- 172.21.15.107:0/862806916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 msgr2=0x7f1f04199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 0x7f1f04199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 -- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04108490 msgr2=0x7f1f0419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04108490 0x7f1f0419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f01e00640 1 -- 172.21.15.107:0/862806916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0009490 con 0x7f1f041033c0 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.672+0000 7f1f02800640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 0x7f1f04199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T20:04:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1f01e00640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef402cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef402dea0 con 0x7f1f041033c0 2024-03-11T20:04:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ef4004350 con 0x7f1f041033c0 2024-03-11T20:04:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4003b20 con 0x7f1f041033c0 2024-03-11T20:04:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef4009560 con 0x7f1f041033c0 2024-03-11T20:04:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.673+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f041a6720 con 0x7f1f041033c0 2024-03-11T20:04:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.674+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04104890 con 0x7f1f041033c0 2024-03-11T20:04:17.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.676+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f1ef4005460 con 0x7f1f041033c0 2024-03-11T20:04:17.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.677+0000 7f1efaa00640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 0x7f1ee80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:17.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.678+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7f1ef40f0550 con 0x7f1f041033c0 2024-03-11T20:04:17.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.678+0000 7f1f01400640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 0x7f1ee80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:17.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.678+0000 7f1f01400640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 0x7f1ee80a40e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f0419ae90 tx=0x7f1ef0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.680+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ef40b90d0 con 0x7f1f041033c0 2024-03-11T20:04:17.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.881+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f0410b6c0 con 0x7f1f041033c0 2024-03-11T20:04:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.883+0000 7f1efaa00640 1 -- 172.21.15.107:0/862806916 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25722 (secure 0 0 0) 0x7f1ef40bc790 con 0x7f1f041033c0 2024-03-11T20:04:17.877 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-11T20:04:17.877 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":107,"fsid":"1ccf666c-dfdf-11ee-95c8-87774f69a715","created":"2024-03-11T19:40:45.251508+0000","modified":"2024-03-11T20:04:09.784998+0000","last_up_change":"2024-03-11T19:48:56.936628+0000","last_in_change":"2024-03-11T19:48:37.164324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:44:40.865282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:49:53.416489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:49:54.674469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T19:49:58.942750+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"aaa69835-f991-4647-8793-70e0f2472ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6803","nonce":2497501905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6805","nonce":2497501905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6809","nonce":2497501905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2497501905},{"type":"v1","addr":"172.21.15.107:6807","nonce":2497501905}]},"public_addr":"172.21.15.107:6803/2497501905","cluster_addr":"172.21.15.107:6805/2497501905","heartbeat_back_addr":"172.21.15.107:6809/2497501905","heartbeat_front_addr":"172.21.15.107:6807/2497501905","state":["exists","up"]},{"osd":1,"uuid":"4de9ffc7-d837-48bc-b205-34fe05f25eab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6801","nonce":4246700402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6803","nonce":4246700402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6807","nonce":4246700402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4246700402},{"type":"v1","addr":"172.21.15.122:6805","nonce":4246700402}]},"public_addr":"172.21.15.122:6801/4246700402","cluster_addr":"172.21.15.122:6803/4246700402","heartbeat_back_addr":"172.21.15.122:6807/4246700402","heartbeat_front_addr":"172.21.15.122:6805/4246700402","state":["exists","up"]},{"osd":2,"uuid":"32cbc8a5-cf58-48bc-8e6e-013627c03c26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6801","nonce":3949304932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6803","nonce":3949304932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6807","nonce":3949304932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3949304932},{"type":"v1","addr":"172.21.15.145:6805","nonce":3949304932}]},"public_addr":"172.21.15.145:6801/3949304932","cluster_addr":"172.21.15.145:6803/3949304932","heartbeat_back_addr":"172.21.15.145:6807/3949304932","heartbeat_front_addr":"172.21.15.145:6805/3949304932","state":["exists","up"]},{"osd":3,"uuid":"fec9056e-4b8e-477a-8004-9f802d9b3808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6811","nonce":3455904746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6813","nonce":3455904746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6817","nonce":3455904746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3455904746},{"type":"v1","addr":"172.21.15.107:6815","nonce":3455904746}]},"public_addr":"172.21.15.107:6811/3455904746","cluster_addr":"172.21.15.107:6813/3455904746","heartbeat_back_addr":"172.21.15.107:6817/3455904746","heartbeat_front_addr":"172.21.15.107:6815/3455904746","state":["exists","up"]},{"osd":4,"uuid":"aafc0f8b-5df2-4a3e-86de-88558c60282e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6809","nonce":1080212908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6811","nonce":1080212908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6815","nonce":1080212908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1080212908},{"type":"v1","addr":"172.21.15.122:6813","nonce":1080212908}]},"public_addr":"172.21.15.122:6809/1080212908","cluster_addr":"172.21.15.122:6811/1080212908","heartbeat_back_addr":"172.21.15.122:6815/1080212908","heartbeat_front_addr":"172.21.15.122:6813/1080212908","state":["exists","up"]},{"osd":5,"uuid":"3144ef43-abab-4f02-b28e-6ac842870975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6809","nonce":3574999361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6811","nonce":3574999361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6815","nonce":3574999361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":3574999361},{"type":"v1","addr":"172.21.15.145:6813","nonce":3574999361}]},"public_addr":"172.21.15.145:6809/3574999361","cluster_addr":"172.21.15.145:6811/3574999361","heartbeat_back_addr":"172.21.15.145:6815/3574999361","heartbeat_front_addr":"172.21.15.145:6813/3574999361","state":["exists","up"]},{"osd":6,"uuid":"474c8a3c-29ec-45d8-8580-952d3bce081a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6819","nonce":1993322562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6821","nonce":1993322562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6825","nonce":1993322562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1993322562},{"type":"v1","addr":"172.21.15.107:6823","nonce":1993322562}]},"public_addr":"172.21.15.107:6819/1993322562","cluster_addr":"172.21.15.107:6821/1993322562","heartbeat_back_addr":"172.21.15.107:6825/1993322562","heartbeat_front_addr":"172.21.15.107:6823/1993322562","state":["exists","up"]},{"osd":7,"uuid":"2ddf2101-a45c-448e-8caa-fab9b4d3bb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6817","nonce":3856508746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6819","nonce":3856508746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6823","nonce":3856508746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":3856508746},{"type":"v1","addr":"172.21.15.122:6821","nonce":3856508746}]},"public_addr":"172.21.15.122:6817/3856508746","cluster_addr":"172.21.15.122:6819/3856508746","heartbeat_back_addr":"172.21.15.122:6823/3856508746","heartbeat_front_addr":"172.21.15.122:6821/3856508746","state":["exists","up"]},{"osd":8,"uuid":"e0816612-8f19-48fe-9290-bebea9bcb91d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6817","nonce":4190884863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6819","nonce":4190884863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6823","nonce":4190884863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":4190884863},{"type":"v1","addr":"172.21.15.145:6821","nonce":4190884863}]},"public_addr":"172.21.15.145:6817/4190884863","cluster_addr":"172.21.15.145:6819/4190884863","heartbeat_back_addr":"172.21.15.145:6823/4190884863","heartbeat_front_addr":"172.21.15.145:6821/4190884863","state":["exists","up"]},{"osd":9,"uuid":"cc8774d5-15de-4134-82a4-fdeb3ff19f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6827","nonce":3311766141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6829","nonce":3311766141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6833","nonce":3311766141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3311766141},{"type":"v1","addr":"172.21.15.107:6831","nonce":3311766141}]},"public_addr":"172.21.15.107:6827/3311766141","cluster_addr":"172.21.15.107:6829/3311766141","heartbeat_back_addr":"172.21.15.107:6833/3311766141","heartbeat_front_addr":"172.21.15.107:6831/3311766141","state":["exists","up"]},{"osd":10,"uuid":"73a69e44-3a8b-4a0e-8516-c0b6f9e02fdd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6825","nonce":1130634331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6827","nonce":1130634331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6831","nonce":1130634331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1130634331},{"type":"v1","addr":"172.21.15.122:6829","nonce":1130634331}]},"public_addr":"172.21.15.122:6825/1130634331","cluster_addr":"172.21.15.122:6827/1130634331","heartbeat_back_addr":"172.21.15.122:6831/1130634331","heartbeat_front_addr":"172.21.15.122:6829/1130634331","state":["exists","up"]},{"osd":11,"uuid":"007f1865-b782-4f5e-98f1-b358a65840f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6825","nonce":2544565646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6827","nonce":2544565646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6831","nonce":2544565646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":2544565646},{"type":"v1","addr":"172.21.15.145:6829","nonce":2544565646}]},"public_addr":"172.21.15.145:6825/2544565646","cluster_addr":"172.21.15.145:6827/2544565646","heartbeat_back_addr":"172.21.15.145:6831/2544565646","heartbeat_front_addr":"172.21.15.145:6829/2544565646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:43:48.740586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:14.125469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:44:37.846014+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:10.838701+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:45:36.942763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:01.540729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:46:36.568911+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:03.371426+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:47:28.033440+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:02.802198+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:29.944956+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:48:54.960294+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2887016959":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6801/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/3157443745":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/927061211":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1620686143":"2024-03-12T19:41:54.175068+0000","172.21.15.107:6800/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/2142594075":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/531077471":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/865925559":"2024-03-12T19:41:19.200454+0000","172.21.15.107:6801/3464148109":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/3318828522":"2024-03-12T19:41:19.200454+0000","172.21.15.107:0/1356730303":"2024-03-12T19:41:54.175068+0000","172.21.15.107:0/1019473398":"2024-03-11T20:58:16.653084+0000","172.21.15.107:0/3959959887":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6801/1546798510":"2024-03-12T19:53:25.384695+0000","172.21.15.107:0/247446141":"2024-03-12T19:53:25.384695+0000","172.21.15.107:6800/1546798510":"2024-03-12T19:53:25.384695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 msgr2=0x7f1ee80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 0x7f1ee80a40e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f0419ae90 tx=0x7f1ef0005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 msgr2=0x7f1f0419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef402cd70 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 shutdown_connections 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f1ee80a1c20 0x7f1ee80a40e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f04199640 0x7f1f04199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f04108490 0x7f1f0419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 --2- 172.21.15.107:0/862806916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041033c0 0x7f1f0419f580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 >> 172.21.15.107:0/862806916 conn(0x7f1f040fec20 msgr2=0x7f1f04100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 shutdown_connections 2024-03-11T20:04:17.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:17.886+0000 7f1f03200640 1 -- 172.21.15.107:0/862806916 wait complete. 2024-03-11T20:04:17.898 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-11T20:04:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 -- 172.21.15.107:0/474070483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff238102b30 msgr2=0x7ff23810b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 --2- 172.21.15.107:0/474070483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff238102b30 0x7ff23810b010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff224009900 tx=0x7ff22402bcc0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 -- 172.21.15.107:0/474070483 shutdown_connections 2024-03-11T20:04:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 --2- 172.21.15.107:0/474070483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff238102b30 0x7ff23810b010 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 --2- 172.21.15.107:0/474070483 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff23810b5e0 0x7ff23810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 --2- 172.21.15.107:0/474070483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff2381025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 -- 172.21.15.107:0/474070483 >> 172.21.15.107:0/474070483 conn(0x7ff2380fb9d0 msgr2=0x7ff2380fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:18.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 -- 172.21.15.107:0/474070483 shutdown_connections 2024-03-11T20:04:18.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.102+0000 7ff23d800640 1 -- 172.21.15.107:0/474070483 wait complete. 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.103+0000 7ff23d800640 1 Processor -- start 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.103+0000 7ff23d800640 1 -- start start 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 0x7ff23819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 0x7ff238199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238199ea0 con 0x7ff238102210 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23819a010 con 0x7ff23810b5e0 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23819a1d0 con 0x7ff238102b30 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 0x7ff238199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39834/0 (socket says 172.21.15.107:39834) 2024-03-11T20:04:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 -- 172.21.15.107:0/3305319610 learned_addr learned my addr 172.21.15.107:0/3305319610 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237400640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 0x7ff23819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 -- 172.21.15.107:0/3305319610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 msgr2=0x7ff23819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 0x7ff23819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 -- 172.21.15.107:0/3305319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 msgr2=0x7ff238199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 0x7ff238199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237e00640 1 -- 172.21.15.107:0/3305319610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224009560 con 0x7ff238102210 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff237400640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 0x7ff23819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.104+0000 7ff23ce00640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 0x7ff238199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff237e00640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9f0 tx=0x7ff22800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228009700 con 0x7ff238102210 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23819a4b0 con 0x7ff238102210 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff22800cd80 con 0x7ff238102210 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2280188c0 con 0x7ff238102210 2024-03-11T20:04:18.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.105+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2381a65d0 con 0x7ff238102210 2024-03-11T20:04:18.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.106+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff238068b60 con 0x7ff238102210 2024-03-11T20:04:18.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.108+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7ff228018a40 con 0x7ff238102210 2024-03-11T20:04:18.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.109+0000 7ff234c00640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 0x7ff21c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.109+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9693+0+0 (secure 0 0 0) 0x7ff228014070 con 0x7ff238102210 2024-03-11T20:04:18.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.110+0000 7ff237400640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 0x7ff21c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.110+0000 7ff237400640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 0x7ff21c0a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff23819af60 tx=0x7ff224005a20 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:18.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.114+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff228052d20 con 0x7ff238102210 2024-03-11T20:04:18.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:17 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/184927130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:17 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3857434588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:17 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/184927130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:17 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3857434588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:17 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/184927130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:17 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3857434588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T20:04:18.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.353+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff238108ad0 con 0x7ff238102210 2024-03-11T20:04:18.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.358+0000 7ff234c00640 1 -- 172.21.15.107:0/3305319610 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff228096a30 con 0x7ff238102210 2024-03-11T20:04:18.351 INFO:teuthology.orchestra.run.smithi107.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-11T20:04:18.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 msgr2=0x7ff21c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 0x7ff21c0a4060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff23819af60 tx=0x7ff224005a20 comp rx=0 tx=0).stop 2024-03-11T20:04:18.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 msgr2=0x7ff23819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e9f0 tx=0x7ff22800eeb0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 shutdown_connections 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7ff21c0a1ba0 0x7ff21c0a4060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff23810b5e0 0x7ff238199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff238102b30 0x7ff23819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 --2- 172.21.15.107:0/3305319610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff238102210 0x7ff23819f650 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 >> 172.21.15.107:0/3305319610 conn(0x7ff2380fb9d0 msgr2=0x7ff2380fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.363+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 shutdown_connections 2024-03-11T20:04:18.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.364+0000 7ff23d800640 1 -- 172.21.15.107:0/3305319610 wait complete. 2024-03-11T20:04:18.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.585+0000 7efe29000640 1 -- 172.21.15.107:0/3626878840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24103f40 msgr2=0x7efe24107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.585+0000 7efe29000640 1 --2- 172.21.15.107:0/3626878840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24103f40 0x7efe24107f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe10009930 tx=0x7efe1002bc30 comp rx=0 tx=0).stop 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 -- 172.21.15.107:0/3626878840 shutdown_connections 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 --2- 172.21.15.107:0/3626878840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24103f40 0x7efe24107f80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 --2- 172.21.15.107:0/3626878840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24108660 0x7efe24110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 --2- 172.21.15.107:0/3626878840 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24103590 0x7efe24103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 -- 172.21.15.107:0/3626878840 >> 172.21.15.107:0/3626878840 conn(0x7efe240fee30 msgr2=0x7efe24101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 -- 172.21.15.107:0/3626878840 shutdown_connections 2024-03-11T20:04:18.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.586+0000 7efe29000640 1 -- 172.21.15.107:0/3626878840 wait complete. 2024-03-11T20:04:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 Processor -- start 2024-03-11T20:04:18.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 -- start start 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24108660 0x7efe2419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24199860 0x7efe24199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419a250 con 0x7efe24103590 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419a3c0 con 0x7efe24199860 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.587+0000 7efe29000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419a580 con 0x7efe24108660 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39878/0 (socket says 172.21.15.107:39878) 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 -- 172.21.15.107:0/416037158 learned_addr learned my addr 172.21.15.107:0/416037158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21400640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24108660 0x7efe2419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe22800640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24199860 0x7efe24199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 -- 172.21.15.107:0/416037158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24108660 msgr2=0x7efe2419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24108660 0x7efe2419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 -- 172.21.15.107:0/416037158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24199860 msgr2=0x7efe24199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24199860 0x7efe24199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 -- 172.21.15.107:0/416037158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10009560 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.588+0000 7efe21e00640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe1400e9f0 tx=0x7efe1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.589+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14009720 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.589+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe241a6570 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.589+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe1400cd80 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.589+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14010690 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.589+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe241a6a50 con 0x7efe24103590 2024-03-11T20:04:18.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.590+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe240683c0 con 0x7efe24103590 2024-03-11T20:04:18.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.592+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7efe140046c0 con 0x7efe24103590 2024-03-11T20:04:18.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.593+0000 7efe1aa00640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 0x7efe080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:18.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.593+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9863+0+0 (secure 0 0 0) 0x7efe14014070 con 0x7efe24103590 2024-03-11T20:04:18.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.594+0000 7efe21400640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 0x7efe080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:18.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.594+0000 7efe21400640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 0x7efe080a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe10000c00 tx=0x7efe10005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:18.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.597+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe14096a50 con 0x7efe24103590 2024-03-11T20:04:18.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a[31707]: 2024-03-11T20:04:18.354+0000 7f1318200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T20:04:18.750 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mds[73130]: -- 172.21.15.107:0/4066999010 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56535fcca1a0 con 0x56535fca0c00 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: starting mds.a at 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mds[66947]: -- 172.21.15.122:0/675535491 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b328a101a0 con 0x55b3289e6800 2024-03-11T20:04:18.751 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: starting mds.b at 2024-03-11T20:04:18.752 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.752 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.752 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mds[66624]: -- 172.21.15.122:0/2834664669 <== mon.2 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b699a041a0 con 0x55b6999dac00 2024-03-11T20:04:18.752 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: starting mds.h at 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mds[67420]: -- 172.21.15.145:0/2004545166 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e952fa1a0 con 0x558e952d0800 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: starting mds.c at 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.776 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mds[67743]: -- 172.21.15.145:0/4146875283 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584584ee1a0 con 0x5584584c4800 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: starting mds.i at 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: ignoring --setuser ceph since I am not root 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: ignoring --setgroup ceph since I am not root 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mds[67112]: -- 172.21.15.145:0/1226011229 <== mon.1 v2:172.21.15.145:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e8f15901a0 con 0x55e8f1566c00 2024-03-11T20:04:18.777 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: starting mds.f at 2024-03-11T20:04:18.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:18.842+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efe2419ad90 con 0x7efe24103590 2024-03-11T20:04:19.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: pgmap v340: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:19.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/862806916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: osdmap e108: 12 total, 12 up, 12 in 2024-03-11T20:04:19.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:18 smithi107 ceph-mon[31712]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T20:04:19.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: pgmap v340: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:19.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/862806916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:19.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T20:04:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T20:04:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T20:04:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T20:04:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: osdmap e108: 12 total, 12 up, 12 in 2024-03-11T20:04:19.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:18 smithi122 ceph-mon[36397]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T20:04:19.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: pgmap v340: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/862806916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/3305319610' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: osdmap e108: 12 total, 12 up, 12 in 2024-03-11T20:04:19.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:18 smithi145 ceph-mon[36730]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T20:04:19.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.360+0000 7efe1aa00640 1 -- 172.21.15.107:0/416037158 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7efe2419ad90 con 0x7efe24103590 2024-03-11T20:04:19.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.364+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 msgr2=0x7efe080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:19.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.364+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 0x7efe080a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe10000c00 tx=0x7efe10005fd0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.364+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 msgr2=0x7efe2419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:19.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.364+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe1400e9f0 tx=0x7efe1400edf0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 shutdown_connections 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7efe080a1ba0 0x7efe080a4060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe24199860 0x7efe24199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efe24108660 0x7efe2419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 --2- 172.21.15.107:0/416037158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe24103590 0x7efe2419f6b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 >> 172.21.15.107:0/416037158 conn(0x7efe240fee30 msgr2=0x7efe241001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 shutdown_connections 2024-03-11T20:04:19.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.365+0000 7efe29000640 1 -- 172.21.15.107:0/416037158 wait complete. 2024-03-11T20:04:19.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-11T20:04:19.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 -- 172.21.15.107:0/517737381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60073e40 msgr2=0x7f4a60111ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 --2- 172.21.15.107:0/517737381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60073e40 0x7f4a60111ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009900 tx=0x7f4a4802bcd0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 -- 172.21.15.107:0/517737381 shutdown_connections 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 --2- 172.21.15.107:0/517737381 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60111ff0 0x7f4a601143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 --2- 172.21.15.107:0/517737381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60073e40 0x7f4a60111ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 --2- 172.21.15.107:0/517737381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 0x7f4a60073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 -- 172.21.15.107:0/517737381 >> 172.21.15.107:0/517737381 conn(0x7f4a600fcaf0 msgr2=0x7f4a600fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 -- 172.21.15.107:0/517737381 shutdown_connections 2024-03-11T20:04:19.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.577+0000 7f4a65c00640 1 -- 172.21.15.107:0/517737381 wait complete. 2024-03-11T20:04:19.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.578+0000 7f4a65c00640 1 Processor -- start 2024-03-11T20:04:19.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.578+0000 7f4a65c00640 1 -- start start 2024-03-11T20:04:19.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 0x7f4a6019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:19.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60073e40 0x7f4a6019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:19.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a601a2bf0 con 0x7f4a60111ff0 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a601a2d60 con 0x7f4a60073520 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a601a2f20 con 0x7f4a60073e40 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39908/0 (socket says 172.21.15.107:39908) 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 -- 172.21.15.107:0/2826306898 learned_addr learned my addr 172.21.15.107:0/2826306898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a57e00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60073e40 0x7f4a6019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a5fe00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 0x7f4a6019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 -- 172.21.15.107:0/2826306898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60073e40 msgr2=0x7f4a6019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60073e40 0x7f4a6019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 -- 172.21.15.107:0/2826306898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 msgr2=0x7f4a6019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 0x7f4a6019dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.579+0000 7f4a65200640 1 -- 172.21.15.107:0/2826306898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009560 con 0x7f4a60111ff0 2024-03-11T20:04:19.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a65200640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800eb40 tx=0x7f4a5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004b20 con 0x7f4a60111ff0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a58004c80 con 0x7f4a60111ff0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a580107b0 con 0x7f4a60111ff0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a601a3200 con 0x7f4a60111ff0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.580+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601aaa80 con 0x7f4a60111ff0 2024-03-11T20:04:19.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.581+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a60067e90 con 0x7f4a60111ff0 2024-03-11T20:04:19.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.583+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f4a58005020 con 0x7f4a60111ff0 2024-03-11T20:04:19.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.585+0000 7f4a5d600640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 0x7f4a440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:19.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.586+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9863+0+0 (secure 0 0 0) 0x7f4a58014070 con 0x7f4a60111ff0 2024-03-11T20:04:19.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.586+0000 7f4a5fe00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 0x7f4a440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:19.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.586+0000 7f4a5fe00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 0x7f4a440a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009bd0 tx=0x7f4a50005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:19.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.590+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a58096d50 con 0x7f4a60111ff0 2024-03-11T20:04:19.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:19.786+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a60104460 con 0x7f4a60111ff0 2024-03-11T20:04:20.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: Cluster is now healthy 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6838/4288858006,v1:172.21.15.122:6839/4288858006] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.122:6836/430661490,v1:172.21.15.122:6837/430661490] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6834/2035102278,v1:172.21.15.145:6835/2035102278] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6837/926535808,v1:172.21.15.145:6839/926535808] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.145:6836/2152156692,v1:172.21.15.145:6838/2152156692] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: mds.? [v2:172.21.15.107:6840/2112804643,v1:172.21.15.107:6841/2112804643] up:boot 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: fsmap 12 up:standby 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T20:04:20.252 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T20:04:20.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T20:04:20.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.253 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:19 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: Cluster is now healthy 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6838/4288858006,v1:172.21.15.122:6839/4288858006] up:boot 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.122:6836/430661490,v1:172.21.15.122:6837/430661490] up:boot 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6834/2035102278,v1:172.21.15.145:6835/2035102278] up:boot 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6837/926535808,v1:172.21.15.145:6839/926535808] up:boot 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.145:6836/2152156692,v1:172.21.15.145:6838/2152156692] up:boot 2024-03-11T20:04:20.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: mds.? [v2:172.21.15.107:6840/2112804643,v1:172.21.15.107:6841/2112804643] up:boot 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: fsmap 12 up:standby 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.277 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:19 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T20:04:20.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T20:04:20.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: Cluster is now healthy 2024-03-11T20:04:20.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/416037158' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6838/4288858006,v1:172.21.15.122:6839/4288858006] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.122:6836/430661490,v1:172.21.15.122:6837/430661490] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6834/2035102278,v1:172.21.15.145:6835/2035102278] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6837/926535808,v1:172.21.15.145:6839/926535808] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.145:6836/2152156692,v1:172.21.15.145:6838/2152156692] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: mds.? [v2:172.21.15.107:6840/2112804643,v1:172.21.15.107:6841/2112804643] up:boot 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: fsmap 12 up:standby 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='mgr.15324 172.21.15.107:0/711163919' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T20:04:20.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:19 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:20.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.364+0000 7f4a5d600640 1 -- 172.21.15.107:0/2826306898 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4a5809a410 con 0x7f4a60111ff0 2024-03-11T20:04:20.357 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' removed 2024-03-11T20:04:20.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 msgr2=0x7f4a440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:20.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 0x7f4a440a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009bd0 tx=0x7f4a50005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:20.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 msgr2=0x7f4a601a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:20.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800eb40 tx=0x7f4a5800c2d0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 shutdown_connections 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f4a440a1ba0 0x7f4a440a4060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a60111ff0 0x7f4a601a25f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a60073e40 0x7f4a6019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 --2- 172.21.15.107:0/2826306898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60073520 0x7f4a6019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 >> 172.21.15.107:0/2826306898 conn(0x7f4a600fcaf0 msgr2=0x7f4a600fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 shutdown_connections 2024-03-11T20:04:20.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.368+0000 7f4a65c00640 1 -- 172.21.15.107:0/2826306898 wait complete. 2024-03-11T20:04:20.386 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-11T20:04:20.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.588+0000 7f0bc6a00640 1 -- 172.21.15.107:0/3234549261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0108660 msgr2=0x7f0bc0110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.588+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/3234549261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0108660 0x7f0bc0110230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009cd0 tx=0x7f0bb802c0e0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 -- 172.21.15.107:0/3234549261 shutdown_connections 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/3234549261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0108660 0x7f0bc0110230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/3234549261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103f40 0x7f0bc0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/3234549261 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0103590 0x7f0bc0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 -- 172.21.15.107:0/3234549261 >> 172.21.15.107:0/3234549261 conn(0x7f0bc00fee30 msgr2=0x7f0bc0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 -- 172.21.15.107:0/3234549261 shutdown_connections 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.589+0000 7f0bc6a00640 1 -- 172.21.15.107:0/3234549261 wait complete. 2024-03-11T20:04:20.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.590+0000 7f0bc6a00640 1 Processor -- start 2024-03-11T20:04:20.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 -- start start 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 0x7f0bc019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0108660 0x7f0bc019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 0x7f0bc0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc019a2c0 con 0x7f0bc0103590 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc019a430 con 0x7f0bc0199810 2024-03-11T20:04:20.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.591+0000 7f0bc6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc019a5a0 con 0x7f0bc0108660 2024-03-11T20:04:20.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bbfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 0x7f0bc019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:20.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 0x7f0bc0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bbfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 0x7f0bc019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39926/0 (socket says 172.21.15.107:39926) 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bbfe00640 1 -- 172.21.15.107:0/910375159 learned_addr learned my addr 172.21.15.107:0/910375159 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bbf400640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0108660 0x7f0bc019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 -- 172.21.15.107:0/910375159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0108660 msgr2=0x7f0bc019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0108660 0x7f0bc019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 -- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 msgr2=0x7f0bc019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 0x7f0bc019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 -- 172.21.15.107:0/910375159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009900 con 0x7f0bc0199810 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.592+0000 7f0bc4a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 0x7f0bc0199cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8037040 tx=0x7f0bb80325e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.593+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8034e20 con 0x7f0bc0199810 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.593+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bb8003e50 con 0x7f0bc0199810 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.593+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004790 con 0x7f0bc0199810 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.593+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc01a64f0 con 0x7f0bc0199810 2024-03-11T20:04:20.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.593+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc01a6920 con 0x7f0bc0199810 2024-03-11T20:04:20.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.594+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0104a60 con 0x7f0bc0199810 2024-03-11T20:04:20.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.596+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7f0bb800a040 con 0x7f0bc0199810 2024-03-11T20:04:20.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.597+0000 7f0bbcc00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 0x7f0ba40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:20.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.598+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9309+0+0 (secure 0 0 0) 0x7f0bb80f09c0 con 0x7f0bc0199810 2024-03-11T20:04:20.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.598+0000 7f0bbfe00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 0x7f0ba40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:20.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.598+0000 7f0bbfe00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 0x7f0ba40a40e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009cd0 tx=0x7f0bb0005f90 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:20.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.600+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0bb80b9810 con 0x7f0bc0199810 2024-03-11T20:04:20.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:20.817+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0bc019ad40 con 0x7f0bc0199810 2024-03-11T20:04:21.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:20 smithi122 ceph-mon[36397]: pgmap v342: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:21.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:20 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:21.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:20 smithi122 ceph-mon[36397]: osdmap e109: 12 total, 12 up, 12 in 2024-03-11T20:04:21.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:20 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/910375159' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.251 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:20 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:20 smithi145 ceph-mon[36730]: pgmap v342: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:21.275 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:20 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:21.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:20 smithi145 ceph-mon[36730]: osdmap e109: 12 total, 12 up, 12 in 2024-03-11T20:04:21.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:20 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/910375159' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.276 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:20 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:20 smithi107 ceph-mon[31712]: pgmap v342: 129 pgs: 129 active+clean; 179 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-11T20:04:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:20 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/2826306898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:20 smithi107 ceph-mon[31712]: osdmap e109: 12 total, 12 up, 12 in 2024-03-11T20:04:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:20 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/910375159' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.283 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:20 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:21.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.382+0000 7f0bbcc00640 1 -- 172.21.15.107:0/910375159 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0bb80440d0 con 0x7f0bc0199810 2024-03-11T20:04:21.375 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' removed 2024-03-11T20:04:21.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.387+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 msgr2=0x7f0ba40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:21.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.387+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 0x7f0ba40a40e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009cd0 tx=0x7f0bb0005f90 comp rx=0 tx=0).stop 2024-03-11T20:04:21.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.387+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 msgr2=0x7f0bc0199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:21.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.387+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 0x7f0bc0199cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8037040 tx=0x7f0bb80325e0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 shutdown_connections 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7f0ba40a1c20 0x7f0ba40a40e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bc0199810 0x7f0bc0199cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0bc0108660 0x7f0bc019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 --2- 172.21.15.107:0/910375159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bc0103590 0x7f0bc019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 >> 172.21.15.107:0/910375159 conn(0x7f0bc00fee30 msgr2=0x7f0bc0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.388+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 shutdown_connections 2024-03-11T20:04:21.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.389+0000 7f0bc6a00640 1 -- 172.21.15.107:0/910375159 wait complete. 2024-03-11T20:04:21.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.625+0000 7fae9d800640 1 -- 172.21.15.107:0/3827060938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae9810d730 msgr2=0x7fae9810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.625+0000 7fae9d800640 1 --2- 172.21.15.107:0/3827060938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae9810d730 0x7fae9810db10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fae88009930 tx=0x7fae8802bc30 comp rx=0 tx=0).stop 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 -- 172.21.15.107:0/3827060938 shutdown_connections 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 --2- 172.21.15.107:0/3827060938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae981145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 --2- 172.21.15.107:0/3827060938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae98107720 0x7fae98107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 --2- 172.21.15.107:0/3827060938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae9810d730 0x7fae9810db10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 -- 172.21.15.107:0/3827060938 >> 172.21.15.107:0/3827060938 conn(0x7fae980756c0 msgr2=0x7fae98077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 -- 172.21.15.107:0/3827060938 shutdown_connections 2024-03-11T20:04:21.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.626+0000 7fae9d800640 1 -- 172.21.15.107:0/3827060938 wait complete. 2024-03-11T20:04:21.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 Processor -- start 2024-03-11T20:04:21.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 -- start start 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae98107720 0x7fae9819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 0x7fae981a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981a2cc0 con 0x7fae98107720 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981a2e30 con 0x7fae981080e0 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.627+0000 7fae9d800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981a2ff0 con 0x7fae9810d730 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae96a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae98107720 0x7fae9819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.107:37530/0 (socket says 172.21.15.107:37530) 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae97400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 0x7fae981a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 -- 172.21.15.107:0/1730565898 learned_addr learned my addr 172.21.15.107:0/1730565898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 -- 172.21.15.107:0/1730565898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 msgr2=0x7fae981a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:21.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 0x7fae981a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 -- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae98107720 msgr2=0x7fae9819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae98107720 0x7fae9819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 -- 172.21.15.107:0/1730565898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009490 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae97400640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 0x7fae981a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.628+0000 7fae8fe00640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ea60 tx=0x7fae8000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.629+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000cde0 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.629+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae800035f0 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.629+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae88009560 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.629+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80010690 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.629+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae981aa970 con 0x7fae981080e0 2024-03-11T20:04:21.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.630+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae980683c0 con 0x7fae981080e0 2024-03-11T20:04:21.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.632+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) v1 ==== 136411+0+0 (secure 0 0 0) 0x7fae80003af0 con 0x7fae981080e0 2024-03-11T20:04:21.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.633+0000 7fae8f400640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 0x7fae7c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:04:21.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.633+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8713+0+0 (secure 0 0 0) 0x7fae80014070 con 0x7fae981080e0 2024-03-11T20:04:21.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.633+0000 7fae96a00640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 0x7fae7c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:04:21.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.634+0000 7fae96a00640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 0x7fae7c0a40e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae88005ab0 tx=0x7fae88005fd0 comp rx=0 tx=0).ready entity=mgr.15324 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:04:21.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.637+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae80096af0 con 0x7fae981080e0 2024-03-11T20:04:21.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:21.844+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae9810fd20 con 0x7fae981080e0 2024-03-11T20:04:22.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.398+0000 7fae8f400640 1 -- 172.21.15.107:0/1730565898 <== mon.2 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v111) v1 ==== 178+0+0 (secure 0 0 0) 0x7fae9810fd20 con 0x7fae981080e0 2024-03-11T20:04:22.397 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data_ec' removed 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.406+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 msgr2=0x7fae7c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.406+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 0x7fae7c0a40e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae88005ab0 tx=0x7fae88005fd0 comp rx=0 tx=0).stop 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.406+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 msgr2=0x7fae9819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.406+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ea60 tx=0x7fae8000ef20 comp rx=0 tx=0).stop 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 shutdown_connections 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:6800/4104906604,v1:172.21.15.107:6801/4104906604] conn(0x7fae7c0a1c20 0x7fae7c0a40e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:22.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae981080e0 0x7fae9819e340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:22.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae9810d730 0x7fae981a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:22.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 --2- 172.21.15.107:0/1730565898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae98107720 0x7fae9819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:04:22.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.407+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 >> 172.21.15.107:0/1730565898 conn(0x7fae980756c0 msgr2=0x7fae98076040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:04:22.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.410+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 shutdown_connections 2024-03-11T20:04:22.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-11T20:04:22.410+0000 7fae9d800640 1 -- 172.21.15.107:0/1730565898 wait complete. 2024-03-11T20:04:22.432 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:04:22.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:04:22.534 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:04:22.570 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-11T20:04:22.570 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T20:04:22.623 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T20:04:22.638 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T20:04:22.655 INFO:tasks.cephadm:Stopping all daemons... 2024-03-11T20:04:22.656 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-11T20:04:22.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a 2024-03-11T20:04:22.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:22.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 ceph-mon[31712]: osdmap e110: 12 total, 12 up, 12 in 2024-03-11T20:04:22.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 ceph-mon[31712]: pgmap v345: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T20:04:22.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 ceph-mon[31712]: from='client.? 172.21.15.107:0/1730565898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:22.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 ceph-mon[31712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:22.751 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:22 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:22.751 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:22 smithi122 ceph-mon[36397]: osdmap e110: 12 total, 12 up, 12 in 2024-03-11T20:04:22.751 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:22 smithi122 ceph-mon[36397]: pgmap v345: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T20:04:22.751 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:22 smithi122 ceph-mon[36397]: from='client.? 172.21.15.107:0/1730565898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:22.751 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:22 smithi122 ceph-mon[36397]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:22.775 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:22 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T20:04:22.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:22 smithi145 ceph-mon[36730]: osdmap e110: 12 total, 12 up, 12 in 2024-03-11T20:04:22.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:22 smithi145 ceph-mon[36730]: pgmap v345: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T20:04:22.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:22 smithi145 ceph-mon[36730]: from='client.? 172.21.15.107:0/1730565898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:22.776 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:22 smithi145 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T20:04:23.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:22 smithi107 systemd[1]: Stopping Ceph mon.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:23.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a[31707]: 2024-03-11T20:04:23.236+0000 7f131f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:23.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a[31707]: 2024-03-11T20:04:23.236+0000 7f131f000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-11T20:04:23.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:23 smithi107 podman[93369]: 2024-03-11 20:04:23.289137913 +0000 UTC m=+0.239273907 container died d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:04:24.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:23 smithi107 podman[93369]: 2024-03-11 20:04:23.803382815 +0000 UTC m=+0.753518781 container cleanup d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-11T20:04:24.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:23 smithi107 bash[93369]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a 2024-03-11T20:04:24.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 11 20:04:24 smithi107 podman[93383]: 2024-03-11 20:04:24.196814368 +0000 UTC m=+0.894436110 container remove d5b6eaa81c1eee55b479c4c940b50e711a490dd88eb5e43aa7873609f97a6fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-a, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-11T20:04:24.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.a.service' 2024-03-11T20:04:24.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:24.833 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-11T20:04:24.834 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-11T20:04:24.834 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.b 2024-03-11T20:04:25.196 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:24 smithi122 systemd[1]: Stopping Ceph mon.b for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:25.466 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b[36393]: 2024-03-11T20:04:25.202+0000 7f4a18e64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:25.466 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b[36393]: 2024-03-11T20:04:25.202+0000 7f4a18e64700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-11T20:04:25.466 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 podman[72974]: 2024-03-11 20:04:25.254661272 +0000 UTC m=+0.191764604 container died 68ea3625f0dfcbf44d72f8b0f75a5547b30a46eb7d14abec1ba524bbe69b6f51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-11T20:04:25.745 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 podman[72974]: 2024-03-11 20:04:25.612226771 +0000 UTC m=+0.549330103 container cleanup 68ea3625f0dfcbf44d72f8b0f75a5547b30a46eb7d14abec1ba524bbe69b6f51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-11T20:04:25.745 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 bash[72974]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b 2024-03-11T20:04:26.001 INFO:journalctl@ceph.mon.b.smithi122.stdout:Mar 11 20:04:25 smithi122 podman[72989]: 2024-03-11 20:04:25.905510406 +0000 UTC m=+0.646506379 container remove 68ea3625f0dfcbf44d72f8b0f75a5547b30a46eb7d14abec1ba524bbe69b6f51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-11T20:04:26.067 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.b.service' 2024-03-11T20:04:26.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:26.114 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-11T20:04:26.114 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-11T20:04:26.115 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.c 2024-03-11T20:04:26.418 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:26 smithi145 systemd[1]: Stopping Ceph mon.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:26.669 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-c[36713]: 2024-03-11T20:04:26.425+0000 7f95afd5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:26.669 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-c[36713]: 2024-03-11T20:04:26.425+0000 7f95afd5b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-11T20:04:26.669 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:26 smithi145 podman[74012]: 2024-03-11 20:04:26.619727204 +0000 UTC m=+0.307781409 container died e60cd6454fc49e50387f439c6c6872f66d1168dbe0dcd58c03ca34f2e99ffdda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:04:27.253 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:27 smithi145 podman[74012]: 2024-03-11 20:04:27.009940922 +0000 UTC m=+0.697995130 container cleanup e60cd6454fc49e50387f439c6c6872f66d1168dbe0dcd58c03ca34f2e99ffdda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-c, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T20:04:27.254 INFO:journalctl@ceph.mon.c.smithi145.stdout:Mar 11 20:04:27 smithi145 bash[74012]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mon-c 2024-03-11T20:04:27.425 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mon.c.service' 2024-03-11T20:04:27.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:27.470 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-11T20:04:27.470 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-11T20:04:27.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x 2024-03-11T20:04:28.032 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:27 smithi107 systemd[1]: Stopping Ceph mgr.x for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:28.518 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T20:04:28.047+0000 7f9bd1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:28.518 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x[31951]: 2024-03-11T20:04:28.047+0000 7f9bd1a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-11T20:04:28.518 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:28 smithi107 podman[93491]: 2024-03-11 20:04:28.140696871 +0000 UTC m=+0.219083919 container died 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T20:04:28.782 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:28 smithi107 podman[93491]: 2024-03-11 20:04:28.709028493 +0000 UTC m=+0.787415578 container cleanup 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:04:28.782 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:28 smithi107 bash[93491]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x 2024-03-11T20:04:29.328 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 11 20:04:29 smithi107 podman[93505]: 2024-03-11 20:04:29.085805086 +0000 UTC m=+0.931474437 container remove 6d84723511ded8e1e85f362d1b52ed848b909029ed15dc3c77133eca76a929e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-11T20:04:29.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.x.service' 2024-03-11T20:04:29.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:29.450 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-11T20:04:29.451 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-11T20:04:29.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.y 2024-03-11T20:04:29.751 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 20:04:29 smithi122 systemd[1]: Stopping Ceph mgr.y for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:30.084 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 20:04:29 smithi122 podman[73105]: 2024-03-11 20:04:29.803888842 +0000 UTC m=+0.144329760 container died 0b1db61399b17fde19bf6f48e8fef1bc7713f8fc51140e60b6770ddc3e5e4fda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307) 2024-03-11T20:04:30.443 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 20:04:30 smithi122 podman[73105]: 2024-03-11 20:04:30.200267079 +0000 UTC m=+0.540707988 container cleanup 0b1db61399b17fde19bf6f48e8fef1bc7713f8fc51140e60b6770ddc3e5e4fda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-11T20:04:30.443 INFO:journalctl@ceph.mgr.y.smithi122.stdout:Mar 11 20:04:30 smithi122 bash[73105]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-y 2024-03-11T20:04:30.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.y.service' 2024-03-11T20:04:30.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:30.651 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-11T20:04:30.651 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-11T20:04:30.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.z 2024-03-11T20:04:30.987 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 20:04:30 smithi145 systemd[1]: Stopping Ceph mgr.z for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:31.276 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 20:04:30 smithi145 podman[74136]: 2024-03-11 20:04:30.994758236 +0000 UTC m=+0.145108535 container died 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-11T20:04:31.769 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 20:04:31 smithi145 podman[74136]: 2024-03-11 20:04:31.424792295 +0000 UTC m=+0.575142571 container cleanup 9847e0d8990b38a883f4a23d4722dd8b9259d48b26ef336aa54bc8a4cc036f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T20:04:31.770 INFO:journalctl@ceph.mgr.z.smithi145.stdout:Mar 11 20:04:31 smithi145 bash[74136]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mgr-z 2024-03-11T20:04:31.973 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mgr.z.service' 2024-03-11T20:04:32.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:32.010 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-11T20:04:32.010 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-11T20:04:32.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.0 2024-03-11T20:04:32.533 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:32 smithi107 systemd[1]: Stopping Ceph osd.0 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:33.032 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0[46193]: 2024-03-11T20:04:32.575+0000 7f0d9f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:33.033 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0[46193]: 2024-03-11T20:04:32.575+0000 7f0d9f400700 -1 osd.0 111 *** Got signal Terminated *** 2024-03-11T20:04:33.033 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0[46193]: 2024-03-11T20:04:32.575+0000 7f0d9f400700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:04:38.303 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:38 smithi107 podman[93624]: 2024-03-11 20:04:38.046062447 +0000 UTC m=+5.649219230 container died 4e96d707160642d497b39481f63302159ad925b83cc08aaf7315e3e0ff9ac352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.license=GPLv2) 2024-03-11T20:04:38.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:38 smithi107 podman[93624]: 2024-03-11 20:04:38.579675182 +0000 UTC m=+6.182831958 container cleanup 4e96d707160642d497b39481f63302159ad925b83cc08aaf7315e3e0ff9ac352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-11T20:04:38.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:38 smithi107 bash[93624]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0 2024-03-11T20:04:39.282 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:38 smithi107 podman[93644]: 2024-03-11 20:04:38.859623505 +0000 UTC m=+0.802106833 container remove 4e96d707160642d497b39481f63302159ad925b83cc08aaf7315e3e0ff9ac352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T20:04:40.032 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:39 smithi107 podman[93732]: 2024-03-11 20:04:39.641281919 +0000 UTC m=+0.037189162 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:04:40.033 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:39 smithi107 podman[93732]: 2024-03-11 20:04:39.774139268 +0000 UTC m=+0.170046540 container create c46406f47bfebfc7c9dad485436dd466b7147c48efa22b31d3bdc2dedad21565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-03-11T20:04:40.509 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:40 smithi107 podman[93732]: 2024-03-11 20:04:40.069411471 +0000 UTC m=+0.465318747 container init c46406f47bfebfc7c9dad485436dd466b7147c48efa22b31d3bdc2dedad21565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-11T20:04:40.509 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:40 smithi107 podman[93732]: 2024-03-11 20:04:40.081161837 +0000 UTC m=+0.477069122 container start c46406f47bfebfc7c9dad485436dd466b7147c48efa22b31d3bdc2dedad21565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-11T20:04:40.509 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:40 smithi107 podman[93732]: 2024-03-11 20:04:40.185030499 +0000 UTC m=+0.580937774 container attach c46406f47bfebfc7c9dad485436dd466b7147c48efa22b31d3bdc2dedad21565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397) 2024-03-11T20:04:40.782 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 11 20:04:40 smithi107 podman[93732]: 2024-03-11 20:04:40.517092426 +0000 UTC m=+0.912999657 container died c46406f47bfebfc7c9dad485436dd466b7147c48efa22b31d3bdc2dedad21565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-0-deactivate, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-11T20:04:41.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.0.service' 2024-03-11T20:04:41.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:41.503 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-11T20:04:41.503 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-11T20:04:41.503 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.1 2024-03-11T20:04:41.804 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:41 smithi122 systemd[1]: Stopping Ceph osd.1 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:42.251 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1[40983]: 2024-03-11T20:04:41.811+0000 7fb5c3b7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:42.251 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1[40983]: 2024-03-11T20:04:41.811+0000 7fb5c3b7b700 -1 osd.1 111 *** Got signal Terminated *** 2024-03-11T20:04:42.251 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1[40983]: 2024-03-11T20:04:41.811+0000 7fb5c3b7b700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:04:47.429 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:47 smithi122 podman[73227]: 2024-03-11 20:04:47.153655757 +0000 UTC m=+5.449652584 container died 55ad9269210c6f02c86aa4075cf2fb2cec8b75e5e02190861a17b6e9f1f177f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-11T20:04:47.751 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:47 smithi122 podman[73227]: 2024-03-11 20:04:47.545707144 +0000 UTC m=+5.841704010 container cleanup 55ad9269210c6f02c86aa4075cf2fb2cec8b75e5e02190861a17b6e9f1f177f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:04:47.751 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:47 smithi122 bash[73227]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1 2024-03-11T20:04:48.087 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:47 smithi122 podman[73244]: 2024-03-11 20:04:47.788535074 +0000 UTC m=+0.628821565 container remove 55ad9269210c6f02c86aa4075cf2fb2cec8b75e5e02190861a17b6e9f1f177f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-11T20:04:48.375 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:47.998657001 +0000 UTC m=+0.018030810 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:04:48.375 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:48.131256633 +0000 UTC m=+0.150630475 container create 73b7cd2ff64545b2c8ab570b0466539714e71528319c422470631dad6b1ef32d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-11T20:04:48.671 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:48.383694338 +0000 UTC m=+0.403068131 container init 73b7cd2ff64545b2c8ab570b0466539714e71528319c422470631dad6b1ef32d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-03-11T20:04:48.671 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:48.387688359 +0000 UTC m=+0.407062150 container start 73b7cd2ff64545b2c8ab570b0466539714e71528319c422470631dad6b1ef32d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-11T20:04:48.671 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:48.475422806 +0000 UTC m=+0.494796613 container attach 73b7cd2ff64545b2c8ab570b0466539714e71528319c422470631dad6b1ef32d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-11T20:04:49.001 INFO:journalctl@ceph.osd.1.smithi122.stdout:Mar 11 20:04:48 smithi122 podman[73335]: 2024-03-11 20:04:48.679946575 +0000 UTC m=+0.699320372 container died 73b7cd2ff64545b2c8ab570b0466539714e71528319c422470631dad6b1ef32d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:04:49.399 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.1.service' 2024-03-11T20:04:49.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:49.442 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-11T20:04:49.442 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-11T20:04:49.442 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.2 2024-03-11T20:04:49.775 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:49 smithi145 systemd[1]: Stopping Ceph osd.2 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:50.275 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T20:04:49.786+0000 7f95707f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:50.275 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T20:04:49.786+0000 7f95707f8700 -1 osd.2 111 *** Got signal Terminated *** 2024-03-11T20:04:50.275 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2[41882]: 2024-03-11T20:04:49.786+0000 7f95707f8700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:04:55.432 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:55 smithi145 podman[74260]: 2024-03-11 20:04:55.171620866 +0000 UTC m=+5.522558381 container died 38fb4eddbd6805cedaefab048f9e85eb8982d6c94c3aa28c5d200569d4e5c05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:04:55.775 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:55 smithi145 podman[74260]: 2024-03-11 20:04:55.523668307 +0000 UTC m=+5.874605814 container cleanup 38fb4eddbd6805cedaefab048f9e85eb8982d6c94c3aa28c5d200569d4e5c05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:04:55.775 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:55 smithi145 bash[74260]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2 2024-03-11T20:04:56.182 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:55 smithi145 podman[74288]: 2024-03-11 20:04:55.859271805 +0000 UTC m=+0.684871607 container remove 38fb4eddbd6805cedaefab048f9e85eb8982d6c94c3aa28c5d200569d4e5c05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-11T20:04:56.471 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.094024668 +0000 UTC m=+0.017588230 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:04:56.471 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.22642728 +0000 UTC m=+0.149990822 container create 6d7691e8a0ac688c24a363aa53ad4322a2f0bc65b70f977fb1811836adce1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397) 2024-03-11T20:04:56.471 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.47932217 +0000 UTC m=+0.402885730 container init 6d7691e8a0ac688c24a363aa53ad4322a2f0bc65b70f977fb1811836adce1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:04:56.761 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.482937162 +0000 UTC m=+0.406500707 container start 6d7691e8a0ac688c24a363aa53ad4322a2f0bc65b70f977fb1811836adce1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:04:56.761 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.57930448 +0000 UTC m=+0.502868044 container attach 6d7691e8a0ac688c24a363aa53ad4322a2f0bc65b70f977fb1811836adce1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:04:57.025 INFO:journalctl@ceph.osd.2.smithi145.stdout:Mar 11 20:04:56 smithi145 podman[74381]: 2024-03-11 20:04:56.770334087 +0000 UTC m=+0.693897654 container died 6d7691e8a0ac688c24a363aa53ad4322a2f0bc65b70f977fb1811836adce1f9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T20:04:57.403 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.2.service' 2024-03-11T20:04:57.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:04:57.446 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-11T20:04:57.447 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-11T20:04:57.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.3 2024-03-11T20:04:58.015 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:04:57 smithi107 systemd[1]: Stopping Ceph osd.3 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:04:58.015 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:04:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:04:58.022+0000 7fee3b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:04:58.015 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:04:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:04:58.022+0000 7fee3b600700 -1 osd.3 111 *** Got signal Terminated *** 2024-03-11T20:04:58.015 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:04:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:04:58.022+0000 7fee3b600700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:04:58.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:04:58 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:04:58.510+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:04:59.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:04:59 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:04:59.489+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:00.275 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:04:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:04:59.788+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:00.501 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:00.254+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:00.520 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:05:00.070+0000 7fee36e00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.282110+0000 front 2024-03-11T20:04:36.283013+0000 (oldest deadline 2024-03-11T20:04:59.683163+0000) 2024-03-11T20:05:00.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:00 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:00.528+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:01.197 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:01 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:05:01.096+0000 7fee36e00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.282110+0000 front 2024-03-11T20:04:36.283013+0000 (oldest deadline 2024-03-11T20:04:59.683163+0000) 2024-03-11T20:05:01.275 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:00.822+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:01.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:01.297+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:01.781 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:01 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:01.522+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:02.167 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:01.808+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:02.471 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:02 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3[52717]: 2024-03-11T20:05:02.064+0000 7fee36e00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.282110+0000 front 2024-03-11T20:04:36.283013+0000 (oldest deadline 2024-03-11T20:04:59.683163+0000) 2024-03-11T20:05:02.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:02.175+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:02.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:02 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:02.259+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:02.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:02 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:02.479+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:03.122 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:02.826+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:03.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:03 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:03.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:03.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:03 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:03.305+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:03.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:03 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:03.433+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:03.782 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:03 smithi107 podman[93835]: 2024-03-11 20:05:03.591771873 +0000 UTC m=+5.722168596 container died f3dc6f8b014da336bc82fc83037ce7826da090cf6ab9139af0f483d5f8b79494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-11T20:05:04.170 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:03 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:03.812+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:04.282 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:04 smithi107 podman[93835]: 2024-03-11 20:05:04.098389995 +0000 UTC m=+6.228786756 container cleanup f3dc6f8b014da336bc82fc83037ce7826da090cf6ab9139af0f483d5f8b79494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-11T20:05:04.282 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:04 smithi107 bash[93835]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3 2024-03-11T20:05:04.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:04.178+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:04.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:04 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:04.320+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:04.782 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:04 smithi107 podman[93867]: 2024-03-11 20:05:04.356612666 +0000 UTC m=+0.750618958 container remove f3dc6f8b014da336bc82fc83037ce7826da090cf6ab9139af0f483d5f8b79494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-11T20:05:04.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:04 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:04.400+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:05.192 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:04.808+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:05.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:05.200+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:05.532 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:05 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:05.413+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:05.532 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:05 smithi107 podman[93956]: 2024-03-11 20:05:05.122021216 +0000 UTC m=+0.033304243 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:05.532 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:05 smithi107 podman[93956]: 2024-03-11 20:05:05.250605868 +0000 UTC m=+0.161888955 container create 6101e9bffceee6c93be4c67f2ed78503c0dfd6b294fe419ef89acf1ce600562a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T20:05:05.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:05 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:05.369+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:06.000 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:05 smithi107 podman[93956]: 2024-03-11 20:05:05.562520576 +0000 UTC m=+0.473803683 container init 6101e9bffceee6c93be4c67f2ed78503c0dfd6b294fe419ef89acf1ce600562a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3-deactivate, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:05:06.001 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:05 smithi107 podman[93956]: 2024-03-11 20:05:05.574925799 +0000 UTC m=+0.486208877 container start 6101e9bffceee6c93be4c67f2ed78503c0dfd6b294fe419ef89acf1ce600562a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-11T20:05:06.001 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:05 smithi107 podman[93956]: 2024-03-11 20:05:05.678190313 +0000 UTC m=+0.589473407 container attach 6101e9bffceee6c93be4c67f2ed78503c0dfd6b294fe419ef89acf1ce600562a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3-deactivate, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-11T20:05:06.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:05.769+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:06.282 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 11 20:05:06 smithi107 podman[93956]: 2024-03-11 20:05:06.008332284 +0000 UTC m=+0.919615355 container died 6101e9bffceee6c93be4c67f2ed78503c0dfd6b294fe419ef89acf1ce600562a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-94a5397) 2024-03-11T20:05:06.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:06.161+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:06.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:06 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:06.400+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:06.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:06 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:06.423+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:06.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.3.service' 2024-03-11T20:05:06.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:06.945 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-11T20:05:06.945 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-11T20:05:06.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.4 2024-03-11T20:05:07.194 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:06.798+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:07.250 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:07 smithi122 systemd[1]: Stopping Ceph osd.4 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:07.455 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:07.202+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:07.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:07 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:07.383+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:07.751 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:07 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:07.281+0000 7f86e2338700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:07.751 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:07 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:07.281+0000 7f86e2338700 -1 osd.4 111 *** Got signal Terminated *** 2024-03-11T20:05:07.751 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:07 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:07.281+0000 7f86e2338700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:07.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:07 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:07.424+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:08.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:07.748+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:08.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:08 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:08.175+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:08.719 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:08 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:08.405+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:08.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:08 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:08.442+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:08.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:08 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:08.377+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:09.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:08 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:08.726+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:09.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:08 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:08.778+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:09.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:08 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:08.778+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:09.500 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:09 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:09.386+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:09.501 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:09 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:09.141+0000 7f86de92c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.406496+0000 front 2024-03-11T20:04:44.406553+0000 (oldest deadline 2024-03-11T20:05:08.506644+0000) 2024-03-11T20:05:09.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:09.171+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:09.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:09 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:09.406+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:09.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:09 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:09.490+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:10.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:09 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:09.704+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:10.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:09.774+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:10.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:09.774+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:10.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:09.906+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:10.425 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:10 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:10.150+0000 7f86de92c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.406496+0000 front 2024-03-11T20:04:44.406553+0000 (oldest deadline 2024-03-11T20:05:08.506644+0000) 2024-03-11T20:05:10.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:10.168+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:10.740 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:10 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:10.432+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:10.740 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:10 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:10.432+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:10.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:10 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:10.533+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:10.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:10 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:10.411+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:11.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:10 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:10.748+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:11.200 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:10.912+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:11.200 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:10.786+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:11.200 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:10.786+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:11.500 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:11 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:11.196+0000 7f86de92c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.406496+0000 front 2024-03-11T20:04:44.406553+0000 (oldest deadline 2024-03-11T20:05:08.506644+0000) 2024-03-11T20:05:11.501 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:11 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:11.400+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:11.501 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:11 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:11.400+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:11.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:11 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:11.208+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:11.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:11 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:11.208+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:11.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:11 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:11.459+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:11.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:11 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:11.524+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:11.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:11 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:11.524+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:12.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:11 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:11.929+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:12.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:11 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:11.769+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:12.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:11 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:11.769+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:12.152 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:11 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:11.780+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:12.428 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:12 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4[47950]: 2024-03-11T20:05:12.160+0000 7f86de92c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.406496+0000 front 2024-03-11T20:04:44.406553+0000 (oldest deadline 2024-03-11T20:05:08.506644+0000) 2024-03-11T20:05:12.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:12.251+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:12.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:12.251+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:12.750 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:12 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:12.436+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:12.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:12 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:12.436+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:12.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:12.483+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:12.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:12.497+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:12.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:12 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:12.497+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:13.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:12.893+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:13.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:12.776+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:13.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:12 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:12.776+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:13.107 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:12 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:12.779+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:13.107 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:12 smithi122 podman[73457]: 2024-03-11 20:05:12.81738719 +0000 UTC m=+5.657120982 container died 60445d508c9d30b36d9bb1fabadc8f8efc67a57341aeabd0cee3d1f1cd469533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T20:05:13.426 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:13 smithi122 podman[73457]: 2024-03-11 20:05:13.19846294 +0000 UTC m=+6.038196767 container cleanup 60445d508c9d30b36d9bb1fabadc8f8efc67a57341aeabd0cee3d1f1cd469533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:13.426 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:13 smithi122 bash[73457]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4 2024-03-11T20:05:13.750 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:13 smithi122 podman[73475]: 2024-03-11 20:05:13.474496924 +0000 UTC m=+0.653538971 container remove 60445d508c9d30b36d9bb1fabadc8f8efc67a57341aeabd0cee3d1f1cd469533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T20:05:13.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:13 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:13.434+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:13.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:13 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:13.434+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:13.769 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:13 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:13.291+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:13.769 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:13 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:13.291+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:13.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:13.438+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:13.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:13.447+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:13.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:13.447+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:14.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:13 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:13.914+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:14.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:13 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:13.778+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:14.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:13 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:13.778+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:14.077 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:13 smithi122 podman[73567]: 2024-03-11 20:05:13.708594984 +0000 UTC m=+0.017240500 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:14.078 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:13 smithi122 podman[73567]: 2024-03-11 20:05:13.828026907 +0000 UTC m=+0.136672409 container create 9627578030bfcfc5ddcf38584d4a825f32fbdc54056a07f7360137b3e0e131b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:14.078 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:13 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:13.810+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:14.368 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:14 smithi122 podman[73567]: 2024-03-11 20:05:14.086264418 +0000 UTC m=+0.394909920 container init 9627578030bfcfc5ddcf38584d4a825f32fbdc54056a07f7360137b3e0e131b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-11T20:05:14.369 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:14 smithi122 podman[73567]: 2024-03-11 20:05:14.089572783 +0000 UTC m=+0.398218292 container start 9627578030bfcfc5ddcf38584d4a825f32fbdc54056a07f7360137b3e0e131b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T20:05:14.369 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:14 smithi122 podman[73567]: 2024-03-11 20:05:14.194623504 +0000 UTC m=+0.503269021 container attach 9627578030bfcfc5ddcf38584d4a825f32fbdc54056a07f7360137b3e0e131b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True) 2024-03-11T20:05:14.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:14 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:14.279+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:14.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:14 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:14.279+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:14.680 INFO:journalctl@ceph.osd.4.smithi122.stdout:Mar 11 20:05:14 smithi122 podman[73567]: 2024-03-11 20:05:14.377517388 +0000 UTC m=+0.686162889 container died 9627578030bfcfc5ddcf38584d4a825f32fbdc54056a07f7360137b3e0e131b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-11T20:05:14.680 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:14 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:14.418+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:14.681 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:14 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:14.418+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:14.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:14 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:14.447+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:14.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:14 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:14.476+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:14.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:14 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:14.476+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:15.001 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:14 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:14.842+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:15.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:14 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:14.898+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:15.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:14 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:14.748+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:15.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:14 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:14.748+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:15.130 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.4.service' 2024-03-11T20:05:15.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:15.174 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-11T20:05:15.175 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-11T20:05:15.175 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.5 2024-03-11T20:05:15.495 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 systemd[1]: Stopping Ceph osd.5 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:15.496 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:15.274+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:15.496 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:15.274+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:15.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:15.467+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:15.751 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:15.467+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:15.775 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:15.503+0000 7f34c30b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:15.775 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:15.503+0000 7f34c30b3700 -1 osd.5 111 *** Got signal Terminated *** 2024-03-11T20:05:15.775 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:15.503+0000 7f34c30b3700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:15.775 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:15.707+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:15.775 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:15.707+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:15.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:15.415+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:15.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:15.498+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:15.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:15 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:15.498+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:16.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:15 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:15.888+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:16.259 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:15.891+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:16.259 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:15 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:15.891+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:16.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:16.266+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:16.527 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:16.266+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:16.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:16.466+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:16.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:16.466+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:16.786 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:16.394+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:16.786 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:16 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:16.394+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:16.888 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:16 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:16.517+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:16.888 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:16 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:16.517+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:17.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:16.891+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:17.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:16.891+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:17.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:16.718+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:17.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:16.718+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:17.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:16 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:16.718+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.336246+0000 front 2024-03-11T20:04:53.336267+0000 (oldest deadline 2024-03-11T20:05:16.236315+0000) 2024-03-11T20:05:17.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:16 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:16.895+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:17.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:17.232+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:17.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:17.232+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:17.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:17.431+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:17.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:17.431+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:17.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:17.393+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:17.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:17 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:17.393+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:17.913 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:17.510+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:17.913 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:17.510+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:17.913 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:17.510+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:18.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:17.907+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:18.026 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:17.907+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:18.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:17.711+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:18.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:17.711+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:18.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:17 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:17.711+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.336246+0000 front 2024-03-11T20:04:53.336267+0000 (oldest deadline 2024-03-11T20:05:16.236315+0000) 2024-03-11T20:05:18.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:17.920+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:18.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:17 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:17.921+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:18.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:18.206+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:18.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:18.206+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:18.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:18.206+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:18.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:18.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:18.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:18.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:18.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:18.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:18.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:18.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:18.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:18 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:18.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:18.870 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:18.526+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:18.870 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:18.526+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:18.870 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:18.526+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:19.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:18.881+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:19.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:18.881+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:19.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:18.707+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:19.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:18.707+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:19.026 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:18 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:18.707+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.336246+0000 front 2024-03-11T20:04:53.336267+0000 (oldest deadline 2024-03-11T20:05:16.236315+0000) 2024-03-11T20:05:19.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:18.878+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:19.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:18 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:18.878+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:19.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:19.242+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:19.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:19.242+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:19.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:19.242+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:19.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:19.386+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:19.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:19.386+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:19.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:19.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:19.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:19.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:19.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:19.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:19.920 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:19.540+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:19.920 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:19.540+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:19.920 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:19.540+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:20.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:19.874+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:20.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:19.874+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:20.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:19.662+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:31.733255+0000 front 2024-03-11T20:04:31.733392+0000 (oldest deadline 2024-03-11T20:04:59.333931+0000) 2024-03-11T20:05:20.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:19.662+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.235039+0000 front 2024-03-11T20:04:45.235031+0000 (oldest deadline 2024-03-11T20:05:08.134944+0000) 2024-03-11T20:05:20.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:19 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5[48717]: 2024-03-11T20:05:19.662+0000 7f34bb69f700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.336246+0000 front 2024-03-11T20:04:53.336267+0000 (oldest deadline 2024-03-11T20:05:16.236315+0000) 2024-03-11T20:05:20.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:19.928+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:20.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:19 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:19.928+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:20.300 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:20.198+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:20.301 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:20.198+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:20.301 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:20.198+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:20.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:20 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:20.342+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:20.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:20 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:20.342+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:20.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:20 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:20.404+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:20.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:20 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:20.404+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:20.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:20 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:20.404+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:20.953 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:20.575+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:20.954 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:20.575+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:20.954 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:20.575+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:21.148 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:21 smithi145 podman[74486]: 2024-03-11 20:05:21.046714588 +0000 UTC m=+5.665152286 container died bab56d11112bdd85779b33b2149e12e1074040ef8da5d4d33efd919e3d760e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-11T20:05:21.148 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:20.834+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:21.148 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:20 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:20.834+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:21.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:20.962+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:21.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:20 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:20.962+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:21.428 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:21.155+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:21.428 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:21.155+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:21.428 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:21.155+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:21.700 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:21 smithi145 podman[74486]: 2024-03-11 20:05:21.43759155 +0000 UTC m=+6.056029259 container cleanup bab56d11112bdd85779b33b2149e12e1074040ef8da5d4d33efd919e3d760e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T20:05:21.701 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:21 smithi145 bash[74486]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5 2024-03-11T20:05:21.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:21.367+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:21.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:21.367+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:21.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:21.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:21.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:21.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:21.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:21 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:21.430+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:21.964 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:21.612+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:21.964 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:21.612+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:21.964 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:21.612+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:22.025 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:21 smithi145 podman[74515]: 2024-03-11 20:05:21.709633617 +0000 UTC m=+0.659642047 container remove bab56d11112bdd85779b33b2149e12e1074040ef8da5d4d33efd919e3d760e85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-11T20:05:22.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:21.831+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:22.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:21 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:21.831+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:22.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:21.972+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:22.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:21 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:21.972+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:22.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:22.147+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:22.147+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:21.953220809 +0000 UTC m=+0.017910039 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:22.065932752 +0000 UTC m=+0.130621953 container create 0de13ae29cdbd863e3e775a81432c0add6d15d86cccd9b0de38eba1b7acdf57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-11T20:05:22.301 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:22.310207655 +0000 UTC m=+0.374896867 container init 0de13ae29cdbd863e3e775a81432c0add6d15d86cccd9b0de38eba1b7acdf57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5-deactivate, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:05:22.595 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:22.313285254 +0000 UTC m=+0.377974463 container start 0de13ae29cdbd863e3e775a81432c0add6d15d86cccd9b0de38eba1b7acdf57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-11T20:05:22.595 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:22.410230437 +0000 UTC m=+0.474919656 container attach 0de13ae29cdbd863e3e775a81432c0add6d15d86cccd9b0de38eba1b7acdf57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-11T20:05:22.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:22.360+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:22.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:22.360+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:22.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:22.443+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:22.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:22.443+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:22.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:22 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:22.443+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:22.938 INFO:journalctl@ceph.osd.5.smithi145.stdout:Mar 11 20:05:22 smithi145 podman[74609]: 2024-03-11 20:05:22.604620353 +0000 UTC m=+0.669309558 container died 0de13ae29cdbd863e3e775a81432c0add6d15d86cccd9b0de38eba1b7acdf57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:22.938 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:22.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:22.938 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:22 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:22.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:22.998 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:22.625+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:22.998 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:22.625+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:22.998 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:22 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:22.625+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:23.213 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:23.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:23.214 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:23.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:23.214 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:23.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:23.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.5.service' 2024-03-11T20:05:23.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:23.006+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:23.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:23.006+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:23.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:23.284 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-11T20:05:23.284 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-11T20:05:23.285 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.6 2024-03-11T20:05:23.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:23.407+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:23.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:23.407+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:23.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 systemd[1]: Stopping Ceph osd.6 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:23.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.451+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:23.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.451+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:23.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.451+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:23.978 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:23.646+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:23.978 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:23.646+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:23.978 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:23.646+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:24.071 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:23.829+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:24.072 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:23 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:23.829+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:24.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:23.987+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:24.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:23 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:23.987+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:24.281 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.940+0000 7f1d3e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:24.282 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.940+0000 7f1d3e000700 -1 osd.6 111 *** Got signal Terminated *** 2024-03-11T20:05:24.282 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:23 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:23.940+0000 7f1d3e000700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:24.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:24 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:24.080+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:24.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:24 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:24.080+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:24.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:24 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:24.080+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:24.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:24.437+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:24.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:24.437+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:24.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:24.428+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:24.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:24.428+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:24.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:24.428+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:24.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:24 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:24.428+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:58.353478+0000 front 2024-03-11T20:04:58.353924+0000 (oldest deadline 2024-03-11T20:05:23.653345+0000) 2024-03-11T20:05:24.943 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:24.630+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:24.944 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:24.630+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:24.944 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:24.630+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:24.944 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:24.630+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:25.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:24.952+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:25.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:24.952+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:25.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:24 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:24.952+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:25.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:25 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:25.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:25.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:25 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:25.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:25.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:25 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:25.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:25.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:24 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:24.859+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:25.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:24 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:24.859+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:25.475+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:25.475+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:25.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:25.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:25.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:25.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:25.421+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:58.353478+0000 front 2024-03-11T20:04:58.353924+0000 (oldest deadline 2024-03-11T20:05:23.653345+0000) 2024-03-11T20:05:25.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:25.664+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:25.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:25.664+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:25.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:25.664+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:25.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:25.664+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:26.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:25.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:26.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:25.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:26.251 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:25 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:25.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:26.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:25 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:25.897+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:26.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:25 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:25.897+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:26.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:26.070+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:26.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:26.070+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:26.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:26.070+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:26.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:26.475+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:26.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:26.475+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:26.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:26.424+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:26.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:26.424+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:26.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:26.424+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:26.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:26 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:26.424+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:58.353478+0000 front 2024-03-11T20:04:58.353924+0000 (oldest deadline 2024-03-11T20:05:23.653345+0000) 2024-03-11T20:05:27.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:26.899+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:27.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:26.899+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:27.000 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:26.899+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:27.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:26.655+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:27.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:26.655+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:27.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:26.655+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:27.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:26 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:26.655+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:27.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:26.882+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:27.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:26 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:26.882+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:27.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:27.050+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:27.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:27.050+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:27.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:27.050+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:27.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:27.442+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:27.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:27.442+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:27.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:27.442+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:27.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:27.442+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:27.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:27.442+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:27.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:27 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:27.442+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:58.353478+0000 front 2024-03-11T20:04:58.353924+0000 (oldest deadline 2024-03-11T20:05:23.653345+0000) 2024-03-11T20:05:27.883 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:27.609+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:27.883 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:27.609+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:27.883 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:27.609+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:27.883 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:27.609+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:28.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:27.891+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:28.251 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:27.891+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:28.251 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:27 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:27.891+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:27.837+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:27.837+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:27 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:27.837+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:28.084+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:28.084+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:28.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:28.084+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:28.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:28.440+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:28.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:28.440+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:28.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:28.440+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:28.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:28.481+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.342413+0000 front 2024-03-11T20:04:35.342493+0000 (oldest deadline 2024-03-11T20:04:58.242617+0000) 2024-03-11T20:05:28.782 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:28.481+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.850685+0000 front 2024-03-11T20:04:44.850500+0000 (oldest deadline 2024-03-11T20:05:10.750465+0000) 2024-03-11T20:05:28.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:28.481+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.751384+0000 front 2024-03-11T20:04:50.751455+0000 (oldest deadline 2024-03-11T20:05:18.352424+0000) 2024-03-11T20:05:28.783 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6[59708]: 2024-03-11T20:05:28.481+0000 7f1d39800700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:58.353478+0000 front 2024-03-11T20:04:58.353924+0000 (oldest deadline 2024-03-11T20:05:23.653345+0000) 2024-03-11T20:05:28.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:28.647+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:28.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:28.647+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:28.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:28.647+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:28.932 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:28.647+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:29.098 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:28.854+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:29.098 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:28.854+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:29.098 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:28 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:28.854+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:29.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:28.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:29.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:28.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:29.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:28 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:28.940+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:29.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:29.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:29.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:29.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:29.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:29.107+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:29.722 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:29 smithi107 podman[94062]: 2024-03-11 20:05:29.387025069 +0000 UTC m=+5.637322144 container died e056f7978a1a54dec81c164abb1b90efb3072db728ad61867772101bc0a11f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:29.722 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:29.421+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:29.722 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:29.421+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:29.722 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:29 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:29.421+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:29.968 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:29.620+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:29.969 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:29.620+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:29.969 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:29.620+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:29.969 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:29.620+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:30.031 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:29 smithi107 podman[94062]: 2024-03-11 20:05:29.915832995 +0000 UTC m=+6.166130109 container cleanup e056f7978a1a54dec81c164abb1b90efb3072db728ad61867772101bc0a11f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-11T20:05:30.032 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:29 smithi107 bash[94062]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6 2024-03-11T20:05:30.140 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:29.855+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:30.140 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:29.855+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:30.140 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:29 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:29.855+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:30.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:29.976+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:30.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:29.976+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:30.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:29 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:29.976+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:30.376 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:30 smithi107 podman[94081]: 2024-03-11 20:05:30.142151077 +0000 UTC m=+0.739008440 container remove e056f7978a1a54dec81c164abb1b90efb3072db728ad61867772101bc0a11f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-11T20:05:30.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:30.149+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:30.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:30.149+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:30.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:30.149+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:30.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:30 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:30.384+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:30.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:30 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:30.384+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:30.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:30 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:30.384+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:31.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:30.622+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:31.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:30.622+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:31.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:30.622+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:31.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:30 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:30.622+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:31.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:30.870+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:31.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:30.870+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:31.025 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:30 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:30.870+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:31.222 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:30 smithi107 podman[94169]: 2024-03-11 20:05:30.757735174 +0000 UTC m=+0.033829547 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:31.222 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:30 smithi107 podman[94169]: 2024-03-11 20:05:30.921666254 +0000 UTC m=+0.197760656 container create 6bfab65d02302c5334f7e37c336ad44060eef4f0554bb3e0f0d1850d2f981c0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:05:31.222 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:31 smithi107 podman[94169]: 2024-03-11 20:05:31.229737285 +0000 UTC m=+0.505831694 container init 6bfab65d02302c5334f7e37c336ad44060eef4f0554bb3e0f0d1850d2f981c0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-11T20:05:31.276 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.026+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:31.276 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.026+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:31.276 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.026+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:31.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:31.156+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:31.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:31.156+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:31.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:31.156+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:31.531 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:31 smithi107 podman[94169]: 2024-03-11 20:05:31.240917193 +0000 UTC m=+0.517011586 container start 6bfab65d02302c5334f7e37c336ad44060eef4f0554bb3e0f0d1850d2f981c0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-03-11T20:05:31.531 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:31 smithi107 podman[94169]: 2024-03-11 20:05:31.387281336 +0000 UTC m=+0.663375771 container attach 6bfab65d02302c5334f7e37c336ad44060eef4f0554bb3e0f0d1850d2f981c0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True) 2024-03-11T20:05:31.531 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:31 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:31.412+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:31.532 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:31 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:31.412+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:31.532 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:31 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:31.412+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:31.987 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:31.650+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:31.988 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:31.650+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:31.988 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:31.650+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:31.988 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:31.650+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:32.031 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 11 20:05:31 smithi107 podman[94199]: 2024-03-11 20:05:31.727354148 +0000 UTC m=+0.036104032 container died 6bfab65d02302c5334f7e37c336ad44060eef4f0554bb3e0f0d1850d2f981c0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-11T20:05:32.134 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:31.843+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:32.135 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:31.844+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:32.135 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:31 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:31.844+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:32.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.995+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:32.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.995+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:32.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:31 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:31.995+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:32.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.6.service' 2024-03-11T20:05:32.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:32.482 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-11T20:05:32.482 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-11T20:05:32.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.7 2024-03-11T20:05:32.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:32.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:32.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:32.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:32.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:32.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:32.613 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:32.442+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:32.613 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:32.442+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:32.613 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:32 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:32.442+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:32.775 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:32 smithi122 systemd[1]: Stopping Ceph osd.7 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:32.776 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:32.600+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:32.776 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:32.600+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:32.776 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:32.600+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:32.776 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:32.600+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:33.090 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:32.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:33.090 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:32.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:33.090 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:32 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:32.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:33.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:32.783+0000 7f1dee252700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:33.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:32.783+0000 7f1dee252700 -1 osd.7 111 *** Got signal Terminated *** 2024-03-11T20:05:33.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:32 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:32.783+0000 7f1dee252700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:33.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:33.020+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:33.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:33.020+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:33.251 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:33.020+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:33.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:33.099+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:33.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:33.099+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:33.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:33.099+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:33.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:33 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:33.449+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:33.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:33 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:33.449+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:33.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:33 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:33.449+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:33.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:33 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:33.449+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:33.996 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:33.558+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:33.997 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:33.558+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:33.997 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:33.558+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:33.997 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:33.558+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:34.118 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:33.848+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:34.118 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:33.848+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:34.118 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:33 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:33.848+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:34.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:34.004+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:34.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:34.004+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:34.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:34.004+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:34.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:34.004+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.495256+0000 front 2024-03-11T20:05:10.495227+0000 (oldest deadline 2024-03-11T20:05:33.395361+0000) 2024-03-11T20:05:34.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:34.127+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:34.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:34.127+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:34.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:34.127+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:34.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:34 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:34.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:34.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:34 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:34.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:34.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:34 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:34.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:34.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:34 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:34.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:35.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:34.595+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:35.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:34.595+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:35.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:34.595+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:35.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:34 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:34.595+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:35.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:34.851+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:35.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:34.851+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:35.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:34.851+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:35.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:34 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:34.851+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:35.500 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:35.011+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:35.500 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:35.011+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:35.500 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:35.011+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:35.500 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:35.011+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.495256+0000 front 2024-03-11T20:05:10.495227+0000 (oldest deadline 2024-03-11T20:05:33.395361+0000) 2024-03-11T20:05:35.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:35.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:35.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:35.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:35.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:35.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:35.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:35 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:35.417+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:35.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:35 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:35.417+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:35.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:35 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:35.417+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:35.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:35 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:35.417+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:36.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:35.633+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:36.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:35.633+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:36.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:35.633+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:36.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:35.633+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:36.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:35.633+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:36.132 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:35.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:36.132 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:35.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:36.132 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:35.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:36.132 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:35 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:35.803+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:36.500 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.022+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:36.501 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.022+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:36.501 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.022+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:36.501 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.022+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.495256+0000 front 2024-03-11T20:05:10.495227+0000 (oldest deadline 2024-03-11T20:05:33.395361+0000) 2024-03-11T20:05:36.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:36.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:36.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:36.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:36.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:36.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:36.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:36.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:36.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:36 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:36.428+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:36.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:36 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:36.428+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:36.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:36 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:36.428+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:36.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:36 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:36.428+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:36.973 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:36.670+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:36.973 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:36.670+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:36.973 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:36.670+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:36.973 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:36.670+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:36.973 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:36.670+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:37.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:36.797+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:37.106 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:36.797+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:37.107 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:36.797+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:37.107 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:36 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:36.797+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:37.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.981+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:42.689554+0000 front 2024-03-11T20:04:42.689579+0000 (oldest deadline 2024-03-11T20:05:07.989693+0000) 2024-03-11T20:05:37.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.981+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.290773+0000 front 2024-03-11T20:04:53.290640+0000 (oldest deadline 2024-03-11T20:05:17.390809+0000) 2024-03-11T20:05:37.250 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.981+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:00.192671+0000 front 2024-03-11T20:05:00.192777+0000 (oldest deadline 2024-03-11T20:05:24.292561+0000) 2024-03-11T20:05:37.251 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:36 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7[54775]: 2024-03-11T20:05:36.981+0000 7f1dea846700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.495256+0000 front 2024-03-11T20:05:10.495227+0000 (oldest deadline 2024-03-11T20:05:33.395361+0000) 2024-03-11T20:05:37.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:37.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:37.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:37.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:37.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:37.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:37.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:37.114+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:37.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:37 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:37.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:37.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:37 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:37.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:37.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:37 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:37.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:37.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:37 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:37.396+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:38.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:37.706+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:38.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:37.706+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:38.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:37.706+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:38.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:37.707+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:38.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:37 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:37.707+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:38.126 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:37.811+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:38.126 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:37.811+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:38.126 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:37.811+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:38.126 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:37 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:37.811+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:38.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:38.134+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:38.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:38.134+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:38.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:38.134+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:38.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:38.134+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:38.576 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:38 smithi122 podman[73685]: 2024-03-11 20:05:38.29108159 +0000 UTC m=+5.618901964 container died cc01bcc8e771e143d10450aaa3d239598552367be044429cbd96b200e30ce362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-11T20:05:38.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:38 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:38.370+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:38.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:38 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:38.370+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:38.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:38 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:38.370+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:38.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:38 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:38.370+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:38.918 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:38 smithi122 podman[73685]: 2024-03-11 20:05:38.684187972 +0000 UTC m=+6.012008346 container cleanup cc01bcc8e771e143d10450aaa3d239598552367be044429cbd96b200e30ce362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:05:38.918 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:38 smithi122 bash[73685]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7 2024-03-11T20:05:38.918 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:38 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:38.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:38.918 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:38 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:38.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:38.918 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:38 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:38.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:38.919 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:38 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:38.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:38.919 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:38 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:38.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:39.227 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:38 smithi122 podman[73702]: 2024-03-11 20:05:38.92720967 +0000 UTC m=+0.630397211 container remove cc01bcc8e771e143d10450aaa3d239598552367be044429cbd96b200e30ce362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:39.096+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:39.096+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:39.096+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:39.096+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:38.861+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:38.861+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:38.861+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:39.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:38 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:38.861+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:39.488 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.236581189 +0000 UTC m=+0.110737015 container create c2662b28e77e8759fdca6b19d6d8abbe217f89bf2d632a085ff3e7462b3eb742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-11T20:05:39.488 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.143323956 +0000 UTC m=+0.017479798 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:39.750 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.497618207 +0000 UTC m=+0.371774057 container init c2662b28e77e8759fdca6b19d6d8abbe217f89bf2d632a085ff3e7462b3eb742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-11T20:05:39.750 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.500943891 +0000 UTC m=+0.375099726 container start c2662b28e77e8759fdca6b19d6d8abbe217f89bf2d632a085ff3e7462b3eb742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-11T20:05:39.751 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.563916805 +0000 UTC m=+0.438072640 container attach c2662b28e77e8759fdca6b19d6d8abbe217f89bf2d632a085ff3e7462b3eb742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-11T20:05:39.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:39 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:39.389+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:39.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:39 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:39.389+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:39.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:39 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:39.389+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:39.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:39 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:39.389+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:40.099 INFO:journalctl@ceph.osd.7.smithi122.stdout:Mar 11 20:05:39 smithi122 podman[73793]: 2024-03-11 20:05:39.790651232 +0000 UTC m=+0.664807084 container died c2662b28e77e8759fdca6b19d6d8abbe217f89bf2d632a085ff3e7462b3eb742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-94a5397) 2024-03-11T20:05:40.099 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:39.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:40.100 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:39.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:40.100 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:39.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:40.100 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:39.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:40.100 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:39.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:40.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:39.832+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:40.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:39.832+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:40.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:39.832+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:40.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:39 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:39.832+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:40.390 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.7.service' 2024-03-11T20:05:40.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:40.430 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-11T20:05:40.430 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-11T20:05:40.430 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.8 2024-03-11T20:05:40.459 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:40.144+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:40.459 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:40.144+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:40.460 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:40.144+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:40.460 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:40.144+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:40.775 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 systemd[1]: Stopping Ceph osd.8 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:40.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:40 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:40.426+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:40.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:40 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:40.426+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:40.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:40 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:40.426+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:40.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:40 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:40.426+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.815+0000 7fd7307de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.815+0000 7fd7307de700 -1 osd.8 111 *** Got signal Terminated *** 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.815+0000 7fd7307de700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.825+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.825+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.825+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:41.122 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:40 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:40.825+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:41.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:40 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:40.795+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:41.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:40 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:40.795+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:41.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:40 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:40.795+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:41.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:40 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:40.795+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:41.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:40 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:40.795+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:41.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:41.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:41.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:41.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:41.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:41.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:41.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:41.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:41.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:41.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:41.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:41 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:41.473+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:41.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:41 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:41.473+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:41.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:41 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:41.473+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:41.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:41 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:41.473+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:42.135 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:41.828+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:42.135 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:41.828+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:42.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:41.828+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:42.136 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:41 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:41.828+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:42.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:41.760+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:42.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:41.760+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:42.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:41.760+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:42.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:41.760+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:42.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:41 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:41.760+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:42.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:42.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:42.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:42.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:42.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:42.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:42.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:42.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:42.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:42.143+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:42.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:42 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:42.498+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:42.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:42 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:42.498+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:42.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:42 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:42.498+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:42.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:42 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:42.498+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:43.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:43.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:43.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:43.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:43.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:43.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:42 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:42.784+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:43.274 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:42.846+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:42.846+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:42.846+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:42 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:42.846+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:43.094+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:43.094+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:43.094+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:43.094+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:43.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:43.094+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:43.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:43 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:43.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:43.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:43 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:43.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:43.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:43 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:43.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:43.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:43 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:43.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:44.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:43.749+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:44.096 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:43.852+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:44.096 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:43.852+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:44.096 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:43.852+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:44.096 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:43.852+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:44.096 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:43 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:43.852+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:19.087836+0000 front 2024-03-11T20:05:19.087890+0000 (oldest deadline 2024-03-11T20:05:43.188016+0000) 2024-03-11T20:05:44.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:44.104+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:44.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:44.104+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:44.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:44.104+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:44.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:44.104+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:44.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:44.104+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:44.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:44 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:44.470+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:44.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:44 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:44.471+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:44.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:44 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:44.471+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:44.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:44 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:44.471+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:44.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:44 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:44.471+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:45.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:45.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:45.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:45.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:45.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:45.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:44 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:44.736+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:44.901+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.581945+0000 front 2024-03-11T20:04:43.582015+0000 (oldest deadline 2024-03-11T20:05:09.481947+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:44.901+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.483540+0000 front 2024-03-11T20:04:53.483561+0000 (oldest deadline 2024-03-11T20:05:15.783822+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:44.901+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:01.585505+0000 front 2024-03-11T20:05:01.585493+0000 (oldest deadline 2024-03-11T20:05:26.885269+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:44.901+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:12.086863+0000 front 2024-03-11T20:05:12.086882+0000 (oldest deadline 2024-03-11T20:05:34.386942+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:44 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8[55756]: 2024-03-11T20:05:44.901+0000 7fd72cdd2700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:19.087836+0000 front 2024-03-11T20:05:19.087890+0000 (oldest deadline 2024-03-11T20:05:43.188016+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:45 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:45.055+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:45 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:45.055+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:45 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:45.055+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:45.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:45 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:45.055+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:45.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:45 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:45.055+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:45.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:45.512+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:45.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:45.512+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:45.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:45.512+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:45.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:45.512+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:45.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:45 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:45.512+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:46.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:46.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:46.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:46.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:46.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:46.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:45 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:45.750+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:46.274 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:46 smithi145 podman[74715]: 2024-03-11 20:05:46.262139574 +0000 UTC m=+5.630955942 container died d38e042dcce4c7e527d99ae8382ae1941c5e07185ab70b2cfde6c505584a2e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:46.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:46 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:46.031+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:46.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:46 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:46.031+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:46.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:46 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:46.031+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:46.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:46 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:46.031+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:46.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:46 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:46.031+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:46.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:46.504+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:46.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:46.504+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:46.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:46.504+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:46.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:46.504+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:46.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:46 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:46.504+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:46.808 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:46 smithi145 podman[74715]: 2024-03-11 20:05:46.60816611 +0000 UTC m=+5.976982470 container cleanup d38e042dcce4c7e527d99ae8382ae1941c5e07185ab70b2cfde6c505584a2e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-11T20:05:46.808 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:46 smithi145 bash[74715]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8 2024-03-11T20:05:47.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:47.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:47.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:47.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:47.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:47.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:46 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:46.725+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:47.069 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:46 smithi145 podman[74734]: 2024-03-11 20:05:46.817659855 +0000 UTC m=+0.551687560 container remove d38e042dcce4c7e527d99ae8382ae1941c5e07185ab70b2cfde6c505584a2e9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-11T20:05:47.328 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.22687702 +0000 UTC m=+0.114004335 container create bb936f5fdb9bf19f8e1fa280b70c9230dc4b4590c132a575aa203776f2588a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-11T20:05:47.328 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.130451783 +0000 UTC m=+0.017579090 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:47.329 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:47 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:47.077+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:47.329 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:47 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:47.077+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:47.329 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:47 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:47.077+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:47.329 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:47 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:47.077+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:47.329 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:47 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:47.077+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:47.732 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.455050892 +0000 UTC m=+0.342178213 container init bb936f5fdb9bf19f8e1fa280b70c9230dc4b4590c132a575aa203776f2588a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T20:05:47.732 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.458121707 +0000 UTC m=+0.345249011 container start bb936f5fdb9bf19f8e1fa280b70c9230dc4b4590c132a575aa203776f2588a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T20:05:47.732 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.545871463 +0000 UTC m=+0.432998769 container attach bb936f5fdb9bf19f8e1fa280b70c9230dc4b4590c132a575aa203776f2588a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-11T20:05:47.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:47.466+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:47.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:47.466+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:47.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:47.466+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:47.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:47.466+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:47.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:47 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:47.466+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:48.002 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:48.003 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:48.003 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:48.003 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:48.003 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:48.003 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:47 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:47.733+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:48.006 INFO:journalctl@ceph.osd.8.smithi145.stdout:Mar 11 20:05:47 smithi145 podman[74823]: 2024-03-11 20:05:47.741296319 +0000 UTC m=+0.628423651 container died bb936f5fdb9bf19f8e1fa280b70c9230dc4b4590c132a575aa203776f2588a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:48.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:48.123+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:48.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:48.123+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:48.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:48.123+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:48.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:48.123+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:48.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:48.123+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:48.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.8.service' 2024-03-11T20:05:48.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:48.423 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-11T20:05:48.423 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-11T20:05:48.423 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.9 2024-03-11T20:05:48.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:48.443+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:48.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:48.443+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:48.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:48.443+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:48.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:48.443+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:48.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:48.443+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:48.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:48 smithi107 systemd[1]: Stopping Ceph osd.9 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:49.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:48 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:48.735+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:49.425 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.074+0000 7f5324200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:49.425 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.074+0000 7f5324200700 -1 osd.9 111 *** Got signal Terminated *** 2024-03-11T20:05:49.425 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.074+0000 7f5324200700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:49.425 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.432+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:49.425 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.432+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:49.426 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.432+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:49.426 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.432+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:49.426 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:49 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:49.432+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:49.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:49.131+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:49.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:49.131+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:49.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:49.131+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:49.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:49.131+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:49.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:49 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:49.131+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:50.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:50.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:50.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:50.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:50.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:50.001 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:49 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:49.758+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:50.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:50.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:50.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:50.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:50.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:50.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:50 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:50.140+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:50.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:50.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:50.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:50.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:50.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:50.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:50.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:50.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:50.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:50 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:50.456+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:51.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:50 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:50.803+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:51.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:51.162+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:51.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:51.436+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:51.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:51.436+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:51.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:51.436+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:51.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:51.436+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:51.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:51 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:51.436+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:52.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:52.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:52.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:52.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:52.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:52.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:52.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:51 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:51.832+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:52.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:52 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:52.177+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:52.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:52 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:52.464+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:52.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:52 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:52.464+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:52.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:52 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:52.464+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:52.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:52 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:52.464+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:52.783 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:52 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:52.464+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:53.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:53.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:53.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:53.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:53.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:53.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:53.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:52 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:52.882+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:53.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:53 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:53.130+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:43.217531+0000 front 2024-03-11T20:04:43.217402+0000 (oldest deadline 2024-03-11T20:05:07.917657+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:50.219848+0000 front 2024-03-11T20:04:50.219738+0000 (oldest deadline 2024-03-11T20:05:15.519936+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:05:02.425109+0000 front 2024-03-11T20:05:02.425216+0000 (oldest deadline 2024-03-11T20:05:28.325391+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:11.827375+0000 front 2024-03-11T20:05:11.827421+0000 (oldest deadline 2024-03-11T20:05:32.927617+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:20.431771+0000 front 2024-03-11T20:05:20.431704+0000 (oldest deadline 2024-03-11T20:05:43.932160+0000) 2024-03-11T20:05:53.782 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:53 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9[66584]: 2024-03-11T20:05:53.461+0000 7f531fa00700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:23.933358+0000 front 2024-03-11T20:05:23.933467+0000 (oldest deadline 2024-03-11T20:05:53.333900+0000) 2024-03-11T20:05:54.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:54.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:54.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:54.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:54.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:54.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:54.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:53 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:53.835+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:54.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:54.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:54.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:54.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:54.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:54.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:54 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:54.124+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:54.895 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:54 smithi107 podman[94285]: 2024-03-11 20:05:54.616773557 +0000 UTC m=+5.716099599 container died c50065bc188da1d15ad6eb5a5129658d57f0fbceac193c92cadee2c3b4828dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:05:55.216 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:55 smithi107 podman[94285]: 2024-03-11 20:05:55.040201917 +0000 UTC m=+6.139527999 container cleanup c50065bc188da1d15ad6eb5a5129658d57f0fbceac193c92cadee2c3b4828dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-11T20:05:55.216 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:55 smithi107 bash[94285]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:55.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:54 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:54.855+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:55.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:55.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:55.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:55.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:55.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:55.526 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:55.136+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:55.531 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:55 smithi107 podman[94303]: 2024-03-11 20:05:55.224655917 +0000 UTC m=+0.592559767 container remove c50065bc188da1d15ad6eb5a5129658d57f0fbceac193c92cadee2c3b4828dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-11T20:05:56.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:56.254 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:55 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:55.838+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:56.261 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:55 smithi107 podman[94389]: 2024-03-11 20:05:55.992516118 +0000 UTC m=+0.119677662 container create 42a237c607bf5a99dfa307eed6a7464586ab9e5e0bde601abb6c91af7aba6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-11T20:05:56.261 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:56 smithi107 podman[94389]: 2024-03-11 20:05:55.910625434 +0000 UTC m=+0.037786953 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:05:56.261 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:56 smithi107 podman[94389]: 2024-03-11 20:05:56.270424346 +0000 UTC m=+0.397585900 container init 42a237c607bf5a99dfa307eed6a7464586ab9e5e0bde601abb6c91af7aba6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-11T20:05:56.523 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:56 smithi107 podman[94389]: 2024-03-11 20:05:56.280125908 +0000 UTC m=+0.407287422 container start 42a237c607bf5a99dfa307eed6a7464586ab9e5e0bde601abb6c91af7aba6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-03-11T20:05:56.524 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:56 smithi107 podman[94389]: 2024-03-11 20:05:56.352801617 +0000 UTC m=+0.479963195 container attach 42a237c607bf5a99dfa307eed6a7464586ab9e5e0bde601abb6c91af7aba6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:56.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:56 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:56.097+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:56.781 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 11 20:05:56 smithi107 podman[94389]: 2024-03-11 20:05:56.715553301 +0000 UTC m=+0.842714808 container died 42a237c607bf5a99dfa307eed6a7464586ab9e5e0bde601abb6c91af7aba6b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:57.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:57.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:56 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:56.887+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:57.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.9.service' 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:57.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:57 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:57.146+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:57.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:05:57.574 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-11T20:05:57.574 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-11T20:05:57.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.10 2024-03-11T20:05:57.872 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 systemd[1]: Stopping Ceph osd.10 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:05:58.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.877+0000 7fdb25982700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:05:58.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.877+0000 7fdb25982700 -1 osd.10 111 *** Got signal Terminated *** 2024-03-11T20:05:58.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.877+0000 7fdb25982700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:05:58.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:58.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:57 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:57.913+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:58.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:58 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:58.166+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:59.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:05:59.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:05:59.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:05:59.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:05:59.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:05:59.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:05:59.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:58 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:58.942+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:05:59.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:05:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:05:59.120+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:00.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:06:00.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:05:59 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:05:59.921+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:06:00.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:00.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:00 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:00.073+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:01.151 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:06:01.152 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:00 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:00.876+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:06:01.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:01.524 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:01.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:01.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:01.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:01.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:01.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:01 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:01.044+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:02.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:35.115398+0000 front 2024-03-11T20:04:35.115309+0000 (oldest deadline 2024-03-11T20:04:59.215162+0000) 2024-03-11T20:06:02.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:44.516222+0000 front 2024-03-11T20:04:44.516068+0000 (oldest deadline 2024-03-11T20:05:09.816295+0000) 2024-03-11T20:06:02.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:52.117282+0000 front 2024-03-11T20:04:52.117315+0000 (oldest deadline 2024-03-11T20:05:17.417157+0000) 2024-03-11T20:06:02.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-11T20:04:59.718483+0000 front 2024-03-11T20:04:59.718337+0000 (oldest deadline 2024-03-11T20:05:23.818061+0000) 2024-03-11T20:06:02.250 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.120124+0000 front 2024-03-11T20:05:10.120162+0000 (oldest deadline 2024-03-11T20:05:34.820243+0000) 2024-03-11T20:06:02.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:14.820688+0000 front 2024-03-11T20:05:14.820747+0000 (oldest deadline 2024-03-11T20:05:41.821175+0000) 2024-03-11T20:06:02.251 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:01 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10[61543]: 2024-03-11T20:06:01.867+0000 7fdb1df6e700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:27.022759+0000 front 2024-03-11T20:05:27.022636+0000 (oldest deadline 2024-03-11T20:05:51.122459+0000) 2024-03-11T20:06:02.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:02.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.006+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:03.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:03.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:03.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:03.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:03.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:03.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:03.276 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:02 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:02.969+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:03.602 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:03 smithi122 podman[73899]: 2024-03-11 20:06:03.314764267 +0000 UTC m=+5.524044559 container died 23328961a8856bdb7c7e20d73125f30e607818af8857f42cfdc643ebbde4201d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-03-11T20:06:03.602 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:03 smithi122 podman[73899]: 2024-03-11 20:06:03.611462663 +0000 UTC m=+5.820742956 container cleanup 23328961a8856bdb7c7e20d73125f30e607818af8857f42cfdc643ebbde4201d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-11T20:06:04.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:03 smithi122 bash[73899]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10 2024-03-11T20:06:04.000 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:03 smithi122 podman[73916]: 2024-03-11 20:06:03.845836867 +0000 UTC m=+0.525468381 container remove 23328961a8856bdb7c7e20d73125f30e607818af8857f42cfdc643ebbde4201d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-11T20:06:04.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:04.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.008+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:04.500 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.071619878 +0000 UTC m=+0.017810739 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:06:04.500 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.171560325 +0000 UTC m=+0.117751190 container create 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-11T20:06:04.500 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.407074908 +0000 UTC m=+0.353265754 container init 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-11T20:06:04.500 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.410537869 +0000 UTC m=+0.356728717 container start 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T20:06:04.967 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.540383317 +0000 UTC m=+0.486574180 container attach 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:06:04.967 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:04 smithi122 podman[74003]: 2024-03-11 20:06:04.693067472 +0000 UTC m=+0.639258319 container died 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, ceph=True) 2024-03-11T20:06:05.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.10.service' 2024-03-11T20:06:05.243 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:05 smithi122 podman[74029]: 2024-03-11 20:06:05.210335548 +0000 UTC m=+0.507313470 container remove 7dcc44540d95762937c587438e923f3a197b748435809fcddc713a4403cd69db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5) 2024-03-11T20:06:05.244 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:05 smithi122 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-11T20:06:05.244 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:05 smithi122 systemd[1]: Stopped Ceph osd.10 for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T20:06:05.244 INFO:journalctl@ceph.osd.10.smithi122.stdout:Mar 11 20:06:05 smithi122 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.10.service: Consumed 57.849s CPU time. 2024-03-11T20:06:05.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:05.259 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-11T20:06:05.259 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-11T20:06:05.260 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.11 2024-03-11T20:06:05.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:05.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:04 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:04.993+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:05.542 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 systemd[1]: Stopping Ceph osd.11 for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:05.943 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.550+0000 7fb6d536c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:05.943 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.550+0000 7fb6d536c700 -1 osd.11 111 *** Got signal Terminated *** 2024-03-11T20:06:05.943 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.550+0000 7fb6d536c700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T20:06:05.943 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:05.944 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:05.944 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:06.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:06.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:06.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:06.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:05 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:05.952+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:07.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:07.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:06 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:06.985+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-03-11T20:05:40.753057+0000 front 2024-03-11T20:05:40.753035+0000 (oldest deadline 2024-03-11T20:06:06.653082+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:08.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:07 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:07.954+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-03-11T20:05:40.753057+0000 front 2024-03-11T20:05:40.753035+0000 (oldest deadline 2024-03-11T20:06:06.653082+0000) 2024-03-11T20:06:09.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:09.274 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:09.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:09 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:09.004+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-03-11T20:05:40.753057+0000 front 2024-03-11T20:05:40.753035+0000 (oldest deadline 2024-03-11T20:06:06.653082+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-11T20:04:36.742504+0000 front 2024-03-11T20:04:36.742255+0000 (oldest deadline 2024-03-11T20:05:02.042268+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6804 osd.1 since back 2024-03-11T20:04:45.443961+0000 front 2024-03-11T20:04:45.443996+0000 (oldest deadline 2024-03-11T20:05:10.743912+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6804 osd.2 since back 2024-03-11T20:04:53.045008+0000 front 2024-03-11T20:04:53.044945+0000 (oldest deadline 2024-03-11T20:05:17.745162+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6812 osd.4 since back 2024-03-11T20:05:10.448777+0000 front 2024-03-11T20:05:10.448713+0000 (oldest deadline 2024-03-11T20:05:35.748716+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6812 osd.5 since back 2024-03-11T20:05:15.749017+0000 front 2024-03-11T20:05:15.749100+0000 (oldest deadline 2024-03-11T20:05:41.049224+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-11T20:05:24.550652+0000 front 2024-03-11T20:05:24.550627+0000 (oldest deadline 2024-03-11T20:05:49.850260+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.122:6820 osd.7 since back 2024-03-11T20:05:33.351405+0000 front 2024-03-11T20:05:33.351483+0000 (oldest deadline 2024-03-11T20:05:58.651273+0000) 2024-03-11T20:06:10.275 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11[62401]: 2024-03-11T20:06:10.020+0000 7fb6d1960700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.145:6820 osd.8 since back 2024-03-11T20:05:40.753057+0000 front 2024-03-11T20:05:40.753035+0000 (oldest deadline 2024-03-11T20:06:06.653082+0000) 2024-03-11T20:06:11.209 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:10 smithi145 podman[74929]: 2024-03-11 20:06:10.962476423 +0000 UTC m=+5.504473670 container died c95738bc1e603705bd975f27254a10e95a0d0b632976a99b6f612783a6e2ea0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-11T20:06:11.492 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:11 smithi145 podman[74929]: 2024-03-11 20:06:11.292284639 +0000 UTC m=+5.834281884 container cleanup c95738bc1e603705bd975f27254a10e95a0d0b632976a99b6f612783a6e2ea0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397) 2024-03-11T20:06:11.492 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:11 smithi145 bash[74929]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11 2024-03-11T20:06:11.774 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:11 smithi145 podman[74948]: 2024-03-11 20:06:11.501710522 +0000 UTC m=+0.542516669 container remove c95738bc1e603705bd975f27254a10e95a0d0b632976a99b6f612783a6e2ea0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-11T20:06:12.179 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:11 smithi145 podman[75037]: 2024-03-11 20:06:11.877475536 +0000 UTC m=+0.092284443 container create 6e894179ddca42cbeff2971efee1db50c20a8c20894f841f452e04b65f8379c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T20:06:12.179 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:11 smithi145 podman[75037]: 2024-03-11 20:06:11.802902171 +0000 UTC m=+0.017711089 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T20:06:12.179 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:12 smithi145 podman[75037]: 2024-03-11 20:06:12.113350517 +0000 UTC m=+0.328159423 container init 6e894179ddca42cbeff2971efee1db50c20a8c20894f841f452e04b65f8379c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397) 2024-03-11T20:06:12.179 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:12 smithi145 podman[75037]: 2024-03-11 20:06:12.116433589 +0000 UTC m=+0.331242508 container start 6e894179ddca42cbeff2971efee1db50c20a8c20894f841f452e04b65f8379c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-11T20:06:12.179 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:12 smithi145 podman[75037]: 2024-03-11 20:06:12.188231938 +0000 UTC m=+0.403040861 container attach 6e894179ddca42cbeff2971efee1db50c20a8c20894f841f452e04b65f8379c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-11T20:06:12.525 INFO:journalctl@ceph.osd.11.smithi145.stdout:Mar 11 20:06:12 smithi145 podman[75037]: 2024-03-11 20:06:12.40388967 +0000 UTC m=+0.618698577 container died 6e894179ddca42cbeff2971efee1db50c20a8c20894f841f452e04b65f8379c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-11T20:06:12.955 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@osd.11.service' 2024-03-11T20:06:13.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:13.037 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-11T20:06:13.037 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-11T20:06:13.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.a 2024-03-11T20:06:13.531 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:13 smithi107 systemd[1]: Stopping Ceph mds.a for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:14.031 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: 2024-03-11T20:06:13.619+0000 7f0c2fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:14.031 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:13 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a[73126]: 2024-03-11T20:06:13.619+0000 7f0c2fc00700 -1 mds.a *** got signal Terminated *** 2024-03-11T20:06:17.932 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:17 smithi107 podman[94510]: 2024-03-11 20:06:17.63364623 +0000 UTC m=+4.140465801 container died 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True) 2024-03-11T20:06:18.281 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:18 smithi107 podman[94510]: 2024-03-11 20:06:18.160927497 +0000 UTC m=+4.667747069 container cleanup 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-11T20:06:18.281 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:18 smithi107 bash[94510]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a 2024-03-11T20:06:18.781 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 11 20:06:18 smithi107 podman[94524]: 2024-03-11 20:06:18.381822857 +0000 UTC m=+0.730959294 container remove 9790c706d881f7a3cb9ab1e7097c92d547d66172750b47caa28c8f241bbf7c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-11T20:06:18.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.a.service' 2024-03-11T20:06:18.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:18.978 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-11T20:06:18.978 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-11T20:06:18.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.d 2024-03-11T20:06:19.281 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:19 smithi107 systemd[1]: Stopping Ceph mds.d for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:19.781 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d[72138]: 2024-03-11T20:06:19.531+0000 7f5bb2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:19.781 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:19 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d[72138]: 2024-03-11T20:06:19.531+0000 7f5bb2800700 -1 mds.d *** got signal Terminated *** 2024-03-11T20:06:23.889 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:23 smithi107 podman[94632]: 2024-03-11 20:06:23.550119926 +0000 UTC m=+4.141040671 container died 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux ) 2024-03-11T20:06:24.234 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:24 smithi107 podman[94632]: 2024-03-11 20:06:24.016840816 +0000 UTC m=+4.607761561 container cleanup 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-11T20:06:24.234 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:24 smithi107 bash[94632]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d 2024-03-11T20:06:24.531 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 11 20:06:24 smithi107 podman[94646]: 2024-03-11 20:06:24.242988926 +0000 UTC m=+0.674695428 container remove 925e0b2b2f4ec86efed616d68233eb719f5e48861b7f75083c4001e565cd2819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:06:24.664 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.d.service' 2024-03-11T20:06:24.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:24.778 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-11T20:06:24.778 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-11T20:06:24.778 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.g 2024-03-11T20:06:25.031 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:24 smithi107 systemd[1]: Stopping Ceph mds.g for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:25.781 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g[72742]: 2024-03-11T20:06:25.309+0000 7f2ccb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:25.781 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:25 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g[72742]: 2024-03-11T20:06:25.309+0000 7f2ccb000700 -1 mds.g *** got signal Terminated *** 2024-03-11T20:06:26.609 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:26 smithi107 podman[94753]: 2024-03-11 20:06:26.339278139 +0000 UTC m=+1.162946141 container died 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-11T20:06:26.912 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:26 smithi107 podman[94753]: 2024-03-11 20:06:26.728552131 +0000 UTC m=+1.552220106 container cleanup 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-11T20:06:26.912 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:26 smithi107 bash[94753]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g 2024-03-11T20:06:27.281 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 11 20:06:26 smithi107 podman[94767]: 2024-03-11 20:06:26.920958783 +0000 UTC m=+0.563748124 container remove 369a10c2519530269a33338c8072c91ee138a67a7c04f43e3a5d0914dd6dc4ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-11T20:06:27.372 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.g.service' 2024-03-11T20:06:27.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:27.488 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-11T20:06:27.488 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-11T20:06:27.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.j 2024-03-11T20:06:28.031 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:27 smithi107 systemd[1]: Stopping Ceph mds.j for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:28.531 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j[71753]: 2024-03-11T20:06:28.195+0000 7f0a4f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:28.531 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:28 smithi107 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j[71753]: 2024-03-11T20:06:28.195+0000 7f0a4f600700 -1 mds.j *** got signal Terminated *** 2024-03-11T20:06:31.784 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:31 smithi107 podman[94872]: 2024-03-11 20:06:31.542336817 +0000 UTC m=+3.542006871 container died 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:06:32.074 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:31 smithi107 podman[94872]: 2024-03-11 20:06:31.898274407 +0000 UTC m=+3.897944500 container cleanup 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-11T20:06:32.075 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:31 smithi107 bash[94872]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j 2024-03-11T20:06:32.503 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 11 20:06:32 smithi107 podman[94898]: 2024-03-11 20:06:32.082395009 +0000 UTC m=+0.529573244 container remove 027681fb6c773d2eed70335900634f96c7e8e05c3cea0a15cbab6948dd73afd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-03-11T20:06:32.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.j.service' 2024-03-11T20:06:32.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:32.655 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-11T20:06:32.655 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-11T20:06:32.655 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.b 2024-03-11T20:06:33.000 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:32 smithi122 systemd[1]: Stopping Ceph mds.b for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:33.499 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: 2024-03-11T20:06:33.043+0000 7f58a5d22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:33.500 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:33 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b[66943]: 2024-03-11T20:06:33.043+0000 7f58a5d22700 -1 mds.b *** got signal Terminated *** 2024-03-11T20:06:34.685 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:34 smithi122 podman[74107]: 2024-03-11 20:06:34.423829165 +0000 UTC m=+1.568999902 container died 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-11T20:06:34.999 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:34 smithi122 podman[74107]: 2024-03-11 20:06:34.818752323 +0000 UTC m=+1.963923054 container cleanup 805b7f0c567810611f6517408a9f86761205169fc5d8ea4bc1fc0c18b576ba63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:06:35.000 INFO:journalctl@ceph.mds.b.smithi122.stdout:Mar 11 20:06:34 smithi122 bash[74107]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-b 2024-03-11T20:06:35.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.b.service' 2024-03-11T20:06:35.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:35.199 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-11T20:06:35.200 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-11T20:06:35.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.e 2024-03-11T20:06:35.500 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:35 smithi122 systemd[1]: Stopping Ceph mds.e for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:35.999 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e[66310]: 2024-03-11T20:06:35.534+0000 7f6315453700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:35.999 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:35 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e[66310]: 2024-03-11T20:06:35.534+0000 7f6315453700 -1 mds.e *** got signal Terminated *** 2024-03-11T20:06:38.972 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:38 smithi122 podman[74239]: 2024-03-11 20:06:38.680218557 +0000 UTC m=+3.256467575 container died 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-11T20:06:39.249 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:39 smithi122 podman[74239]: 2024-03-11 20:06:39.047688421 +0000 UTC m=+3.623937441 container cleanup 1c9c0e4c0a665638b1361c8dc7d13c84148ba22245a13d37eaa3e7874e54b3b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-11T20:06:39.249 INFO:journalctl@ceph.mds.e.smithi122.stdout:Mar 11 20:06:39 smithi122 bash[74239]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-e 2024-03-11T20:06:39.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.e.service' 2024-03-11T20:06:39.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:39.486 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-11T20:06:39.486 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-11T20:06:39.486 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.h 2024-03-11T20:06:39.800 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:39 smithi122 systemd[1]: Stopping Ceph mds.h for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:40.249 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: 2024-03-11T20:06:39.808+0000 7f3b317b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:40.249 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:39 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h[66620]: 2024-03-11T20:06:39.808+0000 7f3b317b3700 -1 mds.h *** got signal Terminated *** 2024-03-11T20:06:42.675 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:42 smithi122 podman[74360]: 2024-03-11 20:06:42.427285379 +0000 UTC m=+2.718201054 container died 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-11T20:06:42.999 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:42 smithi122 podman[74360]: 2024-03-11 20:06:42.784473463 +0000 UTC m=+3.075389134 container cleanup 6a0dd0caeea4973bf0249953dc83a2e998ff1f6bacd537a76f7786d888d2e0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-11T20:06:42.999 INFO:journalctl@ceph.mds.h.smithi122.stdout:Mar 11 20:06:42 smithi122 bash[74360]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-h 2024-03-11T20:06:43.192 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.h.service' 2024-03-11T20:06:43.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:43.226 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-11T20:06:43.226 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-11T20:06:43.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.k 2024-03-11T20:06:43.553 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:43 smithi122 systemd[1]: Stopping Ceph mds.k for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:44.000 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k[65991]: 2024-03-11T20:06:43.562+0000 7fdd78f5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:44.000 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:43 smithi122 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k[65991]: 2024-03-11T20:06:43.562+0000 7fdd78f5c700 -1 mds.k *** got signal Terminated *** 2024-03-11T20:06:47.749 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:47 smithi122 podman[74482]: 2024-03-11 20:06:47.488663956 +0000 UTC m=+4.028277694 container died aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-11T20:06:48.249 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:47 smithi122 podman[74482]: 2024-03-11 20:06:47.938217465 +0000 UTC m=+4.477831204 container cleanup aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-11T20:06:48.249 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:47 smithi122 bash[74482]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k 2024-03-11T20:06:48.250 INFO:journalctl@ceph.mds.k.smithi122.stdout:Mar 11 20:06:48 smithi122 podman[74496]: 2024-03-11 20:06:48.097281409 +0000 UTC m=+0.601857559 container remove aab78686795d5923caa683d6fbb53ceb4b34a8ecec69504a7190e0bb92e12891 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-k, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-11T20:06:48.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.k.service' 2024-03-11T20:06:48.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:48.332 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-11T20:06:48.332 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-11T20:06:48.332 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.c 2024-03-11T20:06:48.671 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:48 smithi145 systemd[1]: Stopping Ceph mds.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:49.026 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: 2024-03-11T20:06:48.680+0000 7f0ed873f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:49.026 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:48 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c[67416]: 2024-03-11T20:06:48.680+0000 7f0ed873f700 -1 mds.c *** got signal Terminated *** 2024-03-11T20:06:50.763 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:50 smithi145 podman[75142]: 2024-03-11 20:06:50.4338074 +0000 UTC m=+1.881383961 container died f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T20:06:51.024 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:50 smithi145 podman[75142]: 2024-03-11 20:06:50.864393334 +0000 UTC m=+2.311969845 container cleanup f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-11T20:06:51.024 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:50 smithi145 bash[75142]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c 2024-03-11T20:06:51.273 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.c.service' 2024-03-11T20:06:51.297 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:51 smithi145 podman[75167]: 2024-03-11 20:06:51.057220346 +0000 UTC m=+0.615468009 container remove f8b8faa9159403c49dd4eba8b4ddacfe36e99b0870f8b05c002416edb4ed7184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-11T20:06:51.297 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:51 smithi145 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.c.service: Deactivated successfully. 2024-03-11T20:06:51.297 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:51 smithi145 systemd[1]: Stopped Ceph mds.c for 1ccf666c-dfdf-11ee-95c8-87774f69a715. 2024-03-11T20:06:51.297 INFO:journalctl@ceph.mds.c.smithi145.stdout:Mar 11 20:06:51 smithi145 systemd[1]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.c.service: Consumed 12.088s CPU time. 2024-03-11T20:06:51.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:51.311 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-11T20:06:51.311 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-11T20:06:51.312 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.f 2024-03-11T20:06:51.625 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:51 smithi145 systemd[1]: Stopping Ceph mds.f for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:52.024 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: 2024-03-11T20:06:51.634+0000 7f1ed5460700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:52.024 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:51 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f[67108]: 2024-03-11T20:06:51.634+0000 7f1ed5460700 -1 mds.f *** got signal Terminated *** 2024-03-11T20:06:54.724 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:54 smithi145 podman[75273]: 2024-03-11 20:06:54.430719058 +0000 UTC m=+2.890722035 container died 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T20:06:54.724 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:54 smithi145 podman[75273]: 2024-03-11 20:06:54.734333873 +0000 UTC m=+3.194336832 container cleanup 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T20:06:55.024 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:54 smithi145 bash[75273]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f 2024-03-11T20:06:55.024 INFO:journalctl@ceph.mds.f.smithi145.stdout:Mar 11 20:06:54 smithi145 podman[75287]: 2024-03-11 20:06:54.977565083 +0000 UTC m=+0.549201883 container remove 965f7b1c5428cc77ab0b89178144f02a2bcec1aa843f3c8ef55fcea7a2b81488 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-03-11T20:06:55.069 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.f.service' 2024-03-11T20:06:55.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:55.104 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-11T20:06:55.104 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-11T20:06:55.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.i 2024-03-11T20:06:55.420 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:55 smithi145 systemd[1]: Stopping Ceph mds.i for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:06:55.774 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: 2024-03-11T20:06:55.429+0000 7f64328e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:06:55.774 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:55 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i[67739]: 2024-03-11T20:06:55.429+0000 7f64328e9700 -1 mds.i *** got signal Terminated *** 2024-03-11T20:06:58.736 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:58 smithi145 podman[75396]: 2024-03-11 20:06:58.432680059 +0000 UTC m=+3.102944159 container died c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-11T20:06:59.024 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:58 smithi145 podman[75396]: 2024-03-11 20:06:58.747023095 +0000 UTC m=+3.417287192 container cleanup c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-11T20:06:59.024 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:58 smithi145 bash[75396]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i 2024-03-11T20:06:59.024 INFO:journalctl@ceph.mds.i.smithi145.stdout:Mar 11 20:06:58 smithi145 podman[75410]: 2024-03-11 20:06:58.9480719 +0000 UTC m=+0.509266284 container remove c396f7974a2688d1a0e3c1d27868b3310e4178b1ffbaf893330413f92652b14b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-i, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:06:59.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.i.service' 2024-03-11T20:06:59.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:06:59.201 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-11T20:06:59.201 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-11T20:06:59.201 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.l 2024-03-11T20:06:59.524 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:06:59 smithi145 systemd[1]: Stopping Ceph mds.l for 1ccf666c-dfdf-11ee-95c8-87774f69a715... 2024-03-11T20:07:00.024 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:06:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l[66797]: 2024-03-11T20:06:59.641+0000 7f2f76dd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T20:07:00.024 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:06:59 smithi145 ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l[66797]: 2024-03-11T20:06:59.641+0000 7f2f76dd3700 -1 mds.l *** got signal Terminated *** 2024-03-11T20:07:03.965 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:07:03 smithi145 podman[75518]: 2024-03-11 20:07:03.647713912 +0000 UTC m=+4.222315652 container died b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-11T20:07:03.965 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:07:03 smithi145 podman[75518]: 2024-03-11 20:07:03.975237632 +0000 UTC m=+4.549839366 container cleanup b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-11T20:07:04.273 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:07:03 smithi145 bash[75518]: ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l 2024-03-11T20:07:04.274 INFO:journalctl@ceph.mds.l.smithi145.stdout:Mar 11 20:07:04 smithi145 podman[75532]: 2024-03-11 20:07:04.159384857 +0000 UTC m=+0.506037524 container remove b159133aaa2bdd83925828efd0b6e49f8a591303f296299bcf778a7f3d21710b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715-mds-l, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T20:07:04.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1ccf666c-dfdf-11ee-95c8-87774f69a715@mds.l.service' 2024-03-11T20:07:04.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T20:07:04.442 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-11T20:07:04.442 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T20:07:04.773 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:07:09.646 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T20:07:09.845 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:07:12.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T20:07:12.509 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:07:15.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:07:15.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:07:15.153 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T20:07:15.182 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-11T20:07:15.185 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi107/crash 2024-03-11T20:07:15.185 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash -- . 2024-03-11T20:07:15.280 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T20:07:15.280 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-03-11T20:07:15.286 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi122/crash 2024-03-11T20:07:15.287 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash -- . 2024-03-11T20:07:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T20:07:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-03-11T20:07:15.331 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi145/crash 2024-03-11T20:07:15.331 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash -- . 2024-03-11T20:07:15.376 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T20:07:15.376 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2024-03-11T20:07:15.377 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-11T20:07:15.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T20:07:15.479 INFO:teuthology.orchestra.run.smithi107.stdout:2024-03-11T19:43:04.898729+0000 mon.a (mon.0) 194 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T20:07:15.480 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-11T20:07:15.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[SEC\]' /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T20:07:15.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]' /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T20:07:15.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[WRN\]' /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-11T20:07:15.733 INFO:teuthology.orchestra.run.smithi107.stdout:2024-03-11T19:43:04.898729+0000 mon.a (mon.0) 194 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T20:07:15.735 INFO:tasks.cephadm:Compressing logs... 2024-03-11T20:07:15.735 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T20:07:15.739 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T20:07:15.741 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:: No such file or directory 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.l.log 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.i.log 2024-03-11T20:07:15.769 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.l.log: /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.11.log 2024-03-11T20:07:15.773 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.z.log 2024-03-11T20:07:15.773 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.8.log 2024-03-11T20:07:15.773 INFO:teuthology.orchestra.run.smithi145.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T20:07:15.773 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.z.log: -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.5.log 2024-03-11T20:07:15.773 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.f.log 2024-03-11T20:07:15.774 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T20:07:15.774 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.f.log: 90.2% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T20:07:15.774 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log 2024-03-11T20:07:15.774 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T20:07:15.774 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.2.log 2024-03-11T20:07:15.777 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log: 94.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.l.log.gz 2024-03-11T20:07:15.777 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.c.log 2024-03-11T20:07:15.777 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.2.log: 87.0% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T20:07:15.777 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.10.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose --/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.10.log: /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.b.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.4.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.1.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.k.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.h.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.k.log: -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.7.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.e.log 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T20:07:15.779 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.y.log 2024-03-11T20:07:15.780 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.k.log.gz 2024-03-11T20:07:15.780 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T20:07:15.782 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.z.log.gz 2024-03-11T20:07:15.782 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.c.log 2024-03-11T20:07:15.783 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.y.log: 94.3% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.e.log.gz 2024-03-11T20:07:15.783 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log 2024-03-11T20:07:15.783 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T20:07:15.783 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T20:07:15.784 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log: 87.0% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T20:07:15.784 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.b.log 2024-03-11T20:07:15.788 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.y.log.gz 2024-03-11T20:07:15.794 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T20:07:15.799 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T20:07:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-11T20:07:15.799 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T20:07:15.800 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.6.log 2024-03-11T20:07:15.801 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.a.log 2024-03-11T20:07:15.802 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T20:07:15.803 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.g.log 2024-03-11T20:07:15.804 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.j.log 2024-03-11T20:07:15.805 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.3.log 2024-03-11T20:07:15.807 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.x.log 2024-03-11T20:07:15.808 INFO:teuthology.orchestra.run.smithi107.stderr: 90.0%/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.3.log: -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T20:07:15.809 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T20:07:15.809 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.0.log 2024-03-11T20:07:15.811 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T20:07:15.812 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5%/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.0.log: -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.g.log.gz 2024-03-11T20:07:15.812 INFO:teuthology.orchestra.run.smithi107.stderr: 94.3% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.j.log.gz 2024-03-11T20:07:15.813 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T20:07:15.814 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log 2024-03-11T20:07:15.815 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T20:07:15.817 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.9.log 2024-03-11T20:07:15.822 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.9.log: 87.0% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T20:07:15.822 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.d.log 2024-03-11T20:07:15.822 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.a.log 2024-03-11T20:07:15.824 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-11T20:07:15.832 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.log: /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.a.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-11T20:07:15.834 INFO:teuthology.orchestra.run.smithi107.stderr: 90.9% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T20:07:15.837 INFO:teuthology.orchestra.run.smithi107.stderr: 94.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.d.log.gz 2024-03-11T20:07:16.119 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.b.log.gz 2024-03-11T20:07:16.176 INFO:teuthology.orchestra.run.smithi107.stderr: 89.8% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mgr.x.log.gz 2024-03-11T20:07:16.393 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.c.log: 91.6% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.c.log.gz 2024-03-11T20:07:16.905 INFO:teuthology.orchestra.run.smithi122.stderr: 94.1% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.h.log.gz 2024-03-11T20:07:17.155 INFO:teuthology.orchestra.run.smithi107.stderr: 90.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mon.a.log.gz 2024-03-11T20:07:17.563 INFO:teuthology.orchestra.run.smithi145.stderr: 95.1% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.f.log.gz 2024-03-11T20:07:19.319 INFO:teuthology.orchestra.run.smithi145.stderr: 95.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.c.log.gz 2024-03-11T20:07:24.857 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.10.log.gz 2024-03-11T20:07:24.946 INFO:teuthology.orchestra.run.smithi145.stderr: 91.8% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.8.log.gz 2024-03-11T20:07:27.603 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.7.log.gz 2024-03-11T20:07:28.003 INFO:teuthology.orchestra.run.smithi122.stderr: 91.9% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.1.log.gz 2024-03-11T20:07:28.363 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.11.log.gz 2024-03-11T20:07:28.881 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.0.log.gz 2024-03-11T20:07:29.390 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.6.log.gz 2024-03-11T20:07:30.571 INFO:teuthology.orchestra.run.smithi145.stderr: 91.6% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.2.log.gz 2024-03-11T20:07:31.057 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.b.log.gz 2024-03-11T20:07:31.155 INFO:teuthology.orchestra.run.smithi122.stderr: 91.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.4.log.gz 2024-03-11T20:07:31.157 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-11T20:07:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m15.402s 2024-03-11T20:07:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:user 1m3.276s 2024-03-11T20:07:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m1.956s 2024-03-11T20:07:31.181 INFO:teuthology.orchestra.run.smithi145.stderr: 93.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.i.log.gz 2024-03-11T20:07:31.264 INFO:teuthology.orchestra.run.smithi107.stderr: 91.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.3.log.gz 2024-03-11T20:07:32.203 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.9.log.gz 2024-03-11T20:07:32.491 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-osd.5.log.gz 2024-03-11T20:07:32.492 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-03-11T20:07:32.492 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m16.737s 2024-03-11T20:07:32.492 INFO:teuthology.orchestra.run.smithi145.stderr:user 1m12.348s 2024-03-11T20:07:32.493 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m2.203s 2024-03-11T20:08:06.649 INFO:teuthology.orchestra.run.smithi107.stderr: 93.4% -- replaced with /var/log/ceph/1ccf666c-dfdf-11ee-95c8-87774f69a715/ceph-mds.a.log.gz 2024-03-11T20:08:06.653 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-03-11T20:08:06.654 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m50.885s 2024-03-11T20:08:06.654 INFO:teuthology.orchestra.run.smithi107.stderr:user 1m43.313s 2024-03-11T20:08:06.654 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m7.442s 2024-03-11T20:08:06.655 INFO:tasks.cephadm:Archiving logs... 2024-03-11T20:08:06.655 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi107/log 2024-03-11T20:08:06.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T20:08:22.617 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi122/log 2024-03-11T20:08:22.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T20:08:32.407 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi145/log 2024-03-11T20:08:32.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T20:08:44.086 INFO:tasks.cephadm:Removing cluster... 2024-03-11T20:08:44.086 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force 2024-03-11T20:08:44.407 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:08:45.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force 2024-03-11T20:08:45.554 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:08:45.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1ccf666c-dfdf-11ee-95c8-87774f69a715 --force 2024-03-11T20:08:46.084 INFO:teuthology.orchestra.run.smithi145.stdout:Deleting cluster with fsid: 1ccf666c-dfdf-11ee-95c8-87774f69a715 2024-03-11T20:08:46.509 INFO:tasks.cephadm:Removing cephadm ... 2024-03-11T20:08:46.509 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T20:08:46.567 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T20:08:46.583 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T20:08:46.599 INFO:tasks.cephadm:Teardown complete 2024-03-11T20:08:46.599 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-11T20:08:46.609 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-11T20:08:46.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T20:08:46.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T20:08:46.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T20:08:46.777 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-11T20:08:46.788 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-11T20:08:46.788 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T20:08:46.797 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T20:08:46.799 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T20:08:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-03-11T20:08:46.813 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-03-11T20:08:46.817 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 8 +247us[ +247us] +/- 159ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 8 377 10 -1902us[-1640us] +/- 60ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +2393us[+2393us] +/- 67ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:^- hv01.front.sepia.ceph.com 2 6 377 6 +396us[ +396us] +/- 159ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -1829us[-1551us] +/- 60ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +2536us[+2814us] +/- 67ms 2024-03-11T20:08:46.818 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T20:08:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +452us[ +452us] +/- 159ms 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -1767us[-1760us] +/- 60ms 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +2609us[+2609us] +/- 67ms 2024-03-11T20:08:46.853 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T20:08:46.854 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-11T20:08:46.865 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-11T20:08:46.866 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-11T20:08:46.876 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T20:08:47.111 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T20:08:47.188 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T20:08:47.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T20:08:47.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:47.355 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T20:08:47.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:47.387 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T20:08:47.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:47.417 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-11T20:08:47.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-11T20:08:47.444 INFO:teuthology.task.internal:Duration was 2243.987974 seconds 2024-03-11T20:08:47.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-11T20:08:47.455 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-11T20:08:47.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T20:08:47.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T20:08:47.465 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T20:08:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T20:08:47.508 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T20:08:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T20:08:48.104 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-11T20:08:48.104 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-03-11T20:08:48.105 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T20:08:48.168 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-03-11T20:08:48.168 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T20:08:48.194 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-03-11T20:08:48.195 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T20:08:48.223 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-11T20:08:48.224 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T20:08:48.230 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T20:08:48.237 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T20:08:48.325 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-11T20:08:48.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T20:08:48.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T20:08:48.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T20:08:48.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-11T20:08:48.883 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-11T20:08:48.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T20:08:48.971 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T20:08:48.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T20:08:49.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-11T20:08:49.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.045 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.047 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.070 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-03-11T20:08:49.092 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-03-11T20:08:49.146 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-03-11T20:08:49.201 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:49.248 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:49.262 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:08:49.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:08:49.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-11T20:08:49.288 INFO:teuthology.task.internal:Transferring archived files... 2024-03-11T20:08:49.289 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi107 2024-03-11T20:08:49.289 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T20:08:49.402 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi122 2024-03-11T20:08:49.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T20:08:49.444 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288/remote/smithi145 2024-03-11T20:08:49.444 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T20:08:49.483 INFO:teuthology.task.internal:Removing archive directory... 2024-03-11T20:08:49.483 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T20:08:49.491 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T20:08:49.493 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T20:08:49.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-11T20:08:49.557 INFO:teuthology.task.internal:Not uploading archives. 2024-03-11T20:08:49.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-11T20:08:49.572 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-11T20:08:49.573 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T20:08:49.581 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T20:08:49.583 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T20:08:49.594 INFO:teuthology.orchestra.run.smithi122.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 20:08 /home/ubuntu/cephtest 2024-03-11T20:08:49.597 INFO:teuthology.orchestra.run.smithi145.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 20:08 /home/ubuntu/cephtest 2024-03-11T20:08:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: 525018 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 20:08 /home/ubuntu/cephtest 2024-03-11T20:08:49.633 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-11T20:08:49.645 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-11T20:08:49.677 INFO:teuthology.nuke:Checking targets against current locks 2024-03-11T20:08:49.702 DEBUG:teuthology.nuke:shortname: smithi107 2024-03-11T20:08:49.703 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T20:08:49.724 DEBUG:teuthology.nuke:shortname: smithi122 2024-03-11T20:08:49.724 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T20:08:49.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.869869', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMG0+8nYDBqhMwFynqAG6lTn/Ot9MVEBFM4ASwNRpNepoPP8kYPiRMBWTdnPJTnboWySGIEy9Zr+/NNFrfvt0eA='} 2024-03-11T20:08:49.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.867203', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnfAqwAXOsOpPpgl8B2XQIvcBw4f+6y+1KDKP5b8IjAVgNtR0fSO30iXeGKFDXN+eYm8IdZQwi8zGhapmj3+y8='} 2024-03-11T20:08:49.751 DEBUG:teuthology.nuke:shortname: smithi145 2024-03-11T20:08:49.752 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T20:08:49.782 INFO:teuthology.orchestra.console.smithi107:Power off 2024-03-11T20:08:49.783 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T20:08:49.792 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:21:47.868639', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBctvLN32amZUN1qWjgvHBMzZ3x/MOQzJaMHMLb915X1aNWK5RqDk31/K2Eunn5zqd1Xtc1FKA3eWadkj9zDooA='} 2024-03-11T20:08:49.806 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-03-11T20:08:49.807 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:49.827 INFO:teuthology.orchestra.console.smithi122:Power off 2024-03-11T20:08:49.827 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T20:08:49.839 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-11T20:08:49.851 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-03-11T20:08:49.851 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:49.869 INFO:teuthology.orchestra.console.smithi145:Power off 2024-03-11T20:08:49.870 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T20:08:49.879 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-03-11T20:08:49.891 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2024-03-11T20:08:49.891 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:49.915 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-11T20:08:53.840 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:53.880 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:53.916 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:53.952 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-11T20:08:53.991 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-03-11T20:08:54.028 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-11T20:08:57.955 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:57.992 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:58.029 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:08:58.069 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-03-11T20:08:58.101 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-03-11T20:08:58.137 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-03-11T20:08:58.171 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-03-11T20:08:58.202 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-03-11T20:08:58.238 INFO:teuthology.orchestra.console.smithi145:Power off completed 2024-03-11T20:08:58.348 INFO:teuthology.run:Summary data: client.0-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} duration: 2243.9879744052887 failure_reason: '"2024-03-11T19:43:04.898729+0000 mon.a (mon.0) 194 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-11T20:08:58.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T20:08:58.446 INFO:teuthology.run:FAIL